Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
HamAdams
Jun 29, 2018

yospos
brb gonna work on my gopherhole IYKWIM :wiggle:

Adbot
ADBOT LOVES YOU

Dans Macabre
Apr 24, 2004


why don't you gopher a walk out of yospos

DaTroof
Nov 16, 2000

CC LIMERICK CONTEST GRAND CHAMPION
There once was a poster named Troof
Who was getting quite long in the toof

nvrgrls posted:

why don't you gopher a walk out of yospos

do you still gopher a good dj mix

HamAdams
Jun 29, 2018

yospos
spun up a gophernicus server this weekend and spent a little time today farting around with it, it's pretty cool and really not that hard to setup

DaTroof
Nov 16, 2000

CC LIMERICK CONTEST GRAND CHAMPION
There once was a poster named Troof
Who was getting quite long in the toof

HamAdams posted:

spun up a gophernicus server this weekend and spent a little time today farting around with it, it's pretty cool and really not that hard to setup

why don't you gophernicus self

armpit_enjoyer
Jan 25, 2023

my god. it's full of posts
https://bugzilla.mozilla.org/show_bug.cgi?id=388195

https://bugzilla.mozilla.org/show_bug.cgi?id=1611668

the duality of man

Athas
Aug 6, 2007

fuck that joker
Let us hope 2024 will be the year of Gopher in the browser.

It is true that Gopherspace is growing.

echinopsis
Apr 13, 2004

by Fluffdaddy
jopher

well-read undead
Dec 13, 2022

Athas
Aug 6, 2007

fuck that joker
A slide from the recent brcon23, demonstrating the growth or decline of Gopherspace:

pre:
## Veronica Statistics

2017: 133 gopher servers, 4.9 mio. unique selectors
2018: 260 gopher servers, 3.7 mio. unique selectors
2019: 320 gopher servers, 4.2 mio. unique selectors
2020: 395 gopher servers, 4.5 mio. unique selectors
2021: 361 gopher servers, 6.0 mio. unique selectors
2022: 325 gopher servers, 5.0 mio. unique selectors
2023: 319 gopher servers, 5.1 mio. unique selectors 

Servers were lost, but selectors is growing again!

A huge difference to 2017, when we started!
Gopherspace is surprisingly volatile. A very dynamic part of the Internet for sure.

Athas fucked around with this message at 13:43 on Aug 22, 2023

BattleMaster
Aug 14, 2000

because of this thread I wrote a gopher server for linux

it uses some fast stuff that didn't exist in the early 90s like epoll and sendfile and port reuse and tcp corking and supports forking into any number of worker processes

I was thinking of going even further and using the cutting edge io_uring stuff to make the fastest gopher server possible

Athas
Aug 6, 2007

fuck that joker

BattleMaster posted:

because of this thread I wrote a gopher server for linux

it uses some fast stuff that didn't exist in the early 90s like epoll and sendfile and port reuse and tcp corking and supports forking into any number of worker processes

I was thinking of going even further and using the cutting edge io_uring stuff to make the fastest gopher server possible

Excellent! Is it available somewhere?

rotor
Jun 11, 2001

classic case of pineapple derangement syndrome
i think the gemini project faq is a good read. As much as I do love old technology, old protocols do have flaws - usually around security and privacy - and I think its important to acknowledge that and create new, modern protocols that at the same time retain as much simplicity as possible given the modern environments.

https://geminiprotocol.net/docs/faq-section-1.gmi

like you've all heard me talk up usenet, and I still think its great, but it has some fuckin issues that it just didnt have back in the early 90s. Spam is endemic and there's no good way to fix it and its largely turned usenet into just a weird file transfer mechanism. I think a modern version of nntp is sorely needed, just like gemini is a modern gopher and its one of the few things about computers i really think about much.

BattleMaster
Aug 14, 2000

Athas posted:

Excellent! Is it available somewhere?

I'll share it when I have a chance to make sure it isn't totally embarrassing and also configurable since it's kind of sub-minimum-viable-product right now - but I do believe it conforms to the RFC at least



it doesn't support CGI or automatically-generated directory listings and I guess I need to figure out how much I care about those things at some point

edit: leaning toward not doing those things, CGI feels like it's against the spirit of gopher and directory listing gopher menus can be generated by scripts that run periodically if they're desired

BattleMaster fucked around with this message at 11:31 on Oct 8, 2023

hbag
Feb 13, 2021

i think i still have a gopherhole somewhere. haven't updated it in years though

Best Bi Geek Squid
Mar 25, 2016

hbag posted:

i think i still have a gopherhole somewhere. haven't updated it in years though

let’s try to keep this thread sfw, thanks

Athas
Aug 6, 2007

fuck that joker

rotor posted:

i think the gemini project faq is a good read. As much as I do love old technology, old protocols do have flaws - usually around security and privacy - and I think its important to acknowledge that and create new, modern protocols that at the same time retain as much simplicity as possible given the modern environments.

I think that is missing the point. Don't put medical data on Gopher and don't expect your bank to make a Gopher interface. I don't mind gophers (the encrypted version), but for me gopher is a way to LARP a simpler internet where all that complexity is not needed. I don't need everything to be on Gopher, I just need to have nerd fun on Gopher, and I don't need complex security and privacy for that.

Gemini is fine, but there is less stuff on Gemini, so I have not investigated it much. Although I noticed that my favourite Gopher client does support it.

Achmed Jones
Oct 16, 2004



nah, integrity (and confidentiality) of traffic is pretty crucial even to throw-away silly data. if you think tls is only needed for banking and medical data, that reflects more on your understanding of what that technology provides than on the one of 35 years ago.

Cybernetic Vermin
Apr 18, 2005

probably most importantly pretty good to be clear on whether this is all a funny tech joke that should absolutely not be used in any context where the word "use" is appropriate, or a real thing.

because for anything even approaching use defaulting to an implementation done by one guy gleefully applying every available low-level trick available turns from fun and cool to real bad quick.

Athas
Aug 6, 2007

fuck that joker

Achmed Jones posted:

nah, integrity (and confidentiality) of traffic is pretty crucial even to throw-away silly data. if you think tls is only needed for banking and medical data, that reflects more on your understanding of what that technology provides than on the one of 35 years ago.

It's not very important to anything I use it for :shrug:

For me Gopher is not a tech joke, but it is a tech I use when I don't want to worry about all the complex stuff of modern real world protocols.

Athas fucked around with this message at 18:22 on Oct 8, 2023

rotor
Jun 11, 2001

classic case of pineapple derangement syndrome

Athas posted:

I think that is missing the point. Don't put medical data on Gopher and don't expect your bank to make a Gopher interface. I don't mind gophers (the encrypted version), but for me gopher is a way to LARP a simpler internet where all that complexity is not needed. I don't need everything to be on Gopher, I just need to have nerd fun on Gopher, and I don't need complex security and privacy for that.

I know the faq is long but if youre interested in low-power/appropriate technologies you should give it a read, esp the parts where they justify their tech decisions and their philosophy.

rotor
Jun 11, 2001

classic case of pineapple derangement syndrome
tl;dr is that its less about someone stealing your bus pass and more about surveillance and hijacked content

BattleMaster
Aug 14, 2000

Athas posted:

Excellent! Is it available somewhere?

now it is, after a huge amount of work since my last post when it was even less of a minimum viable product

https://github.com/theothersarah/sgopher

warning: this is the hobbyest of hobbyist projects programmed by someone who looked at some websites of I/O multiplexing and not the code of actual servers, so it may be literal poop from an rear end or at least outsider art. Also note that it is mega linux and uses the most arcane linux-specific things I could find and needs a kernel in the neighborhood of 5.10 or so and liburing-dev to compile

It is configured entirely via the command line, and it does not have throttling or logging, it doesn't have TLS support, and it doesn't have automatic directory listings, but it does have CGI with query strings to support gopher menu type 7 or dynamic content

It implements the entirety of the original RFC as close as written as I could except that:
1. It doesn't append the period CRLF to the end of a gophermap. It's expected that you put that there in the file
2. It doesn't reprocess text files to use CRLF line endings, escape periods at the start of a line, or add the ending period CRLF. I tried this and it just slowed it down for clients that didn't give a poo poo anyway or didn't understand and would incorrectly display the added things, so text is transmitted exactly the same as other files

It also insists that the client send CRLF in the request as per the RFC so at least one client, bombadillo, gets rejected because it only adds LF. I arrogantly feel like clients should follow the spec so I don't know if I'll ever bother making it worth both ways

I can't claim it's secure but it does the following things to try to be secure:

1. It opens up the serving directory and opens all files relative to that using openat
2. It breaks the request string down into path components and rejects it if any components begin with periods. This is intended to stop the use of relative paths to break out of the serving directory but it has the side effect on not allowing access to hidden files, which like, yeah that makes sense
3. If the path is accepted, it prepends ./ to it to force it to be a relative path to stop people from bypassing openat with absolute paths which causes it to ignore the directory. this works even if the request begins with a / since linux ignores several /s in a row
4. after opening the tile, it will only proceed if the resulting file is a directory or regular file, and if it's world readable
5. If it's a directory, it tries to open a gophermap inside it. It will loop if gophermap in turn is a directory until it finds something. (This means there is an infinite loop if you make a gophermap that is a symlink to its own directory. I will not stop you from doing that but maybe don't)
6. Once it arrives at a file, if it's world executable it forks off a process, dup2s the socket over STDOUT_FILENO, and fexecves the file, passing the query string as a command line argument. If it's not executable, it begins sending it via sendfile, using I/O multiplexing via epoll to service other requests in between sending chunks of the file to you

I'm pretty proud of how I handle the CGI (unless there's some glaring security flaw beyond the obvious one that results from letting people execute programs on your computer, at which point I'll disown it) because I use a raw syscall to clone to get a pidfd from it (instead of having to do a fork/pidfd_open, at which point what do you even do if the fork works but the pidfd_open fails for some reason) and then monitor the pidfd for exiting to do the final client cleanup. Also while it uses timestamps on the last interaction to handle timeouts during normal file transfers, it can't do that in the same way so instead it uses getsockopt to get TCP info from the socket (which it didn't close after the fork for this reason) and spy on the time of last outgoing transfer, killing the process (and then doing client cleanup after the process ends and the pidfd returns) if the outgoing connection was idle for too long

it also includes a test tool that slams a gopher server (please for the love of god only use it on your own for gently caress's sake) to benchmark it, and I'm able to get over 20,000 requests per second on small files or saturate the gigabit ethernet on larger files using a single process running on a 12 year old computer, so I'd say it's probably a pretty fast implementation

while it withstands all my tests and attempts at abuse the creaky questionable part is the event loop that uses epoll with io_uring to batch epollctl and close calls. I originally did it with just plain epoll_ctl calls but I figured batching this stuff would be an easy way to get a bit more performance since proposals for batched epoll_ctl calls never made it into linux in spite of work done on them. Also the io_uring guy hates epoll and tried to remove epoll_ctl support from it so I kind of wanted to spite him just a little since I really like epoll

however, one day I'll work on a version of this that actually purely does use io_uring because that's even faster, although gently caress you io_uring guy "I'm not implementing sendfile because you can do it with pipes and existing io_uring calls" how about I don't want to do it that way thanks

anyway yeah I had a lot of fun writing this, I'll do further work on it gradually, and I hope it isn't insecure although I can't say it isn't so maybe don't use it on the internet

edit: known issue: the waitid in the parent process supervisor fails as if I ignored SIGCHLD in the parent, but I didn't and the child process still becomes a zombie and isn't automatically reaped as if I ignored SIGCHLD. Honestly don't know, maybe something's wrong with using it with a pidfd in linux 5.15, or I'm using it in a subtly wrong way, but it doesn't come into play during normal operation except during closing down anyway (or if someone sends SIGTERM to a child)

BattleMaster fucked around with this message at 17:14 on May 4, 2024

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

hell yes, gopher server with io_uring

hbag
Feb 13, 2021

yospos gopher server when tbh

Ocean of Milk
Jun 25, 2018

oh yeah
Someone on reddit once made a comment that resonated with me (yes that is possible):

quote:

quote:

Semantic HTML is great, use it!
Yeah. People say the semantic web "failed" but that's only because some of the biggest money-making sites are ad funnels where the actual content is just the lamp on the anglerfish. They don't want to be readable, or indexable, or linkable, so they never will be. They'll turn into DRM'd videos of text if they have to, to force you to see ads.
If you actually want your site to be readable, accessible, fast, light, and offline-friendly, HTML can do that.
I think the Gopher / Gemini revival is pretty misguided in this - They aren't gonna convince the anglerfish, and if you're already working in good faith, you could just write better HTML. It's the equivalent of choosing x86 assembly to avoid a problem with C++98's type system. It's also throwing the HTTP baby out with the web bathwater. HTTP(S) 1.1 and 3 are really nice protocols.
By no means do I like HTML - It's the worst part of the web stack, and when I saw that "I'm converting my site to PDF" I had to seriously wonder if they were onto something - But I can put up with it while it's around.

I think we see some of that with the advent of the "indie web" and webrings (example https://webring.recurse.com/)

Captain Foo
May 11, 2004

we vibin'
we slidin'
we breathin'
we dyin'

Subjunctive posted:

hell yes, gopher server with io_uring

yospos as fuuuuuuuck

rotor
Jun 11, 2001

classic case of pineapple derangement syndrome

Ocean of Milk posted:

Someone on reddit once made a comment that resonated with me (yes that is possible):

I think we see some of that with the advent of the "indie web" and webrings (example https://webring.recurse.com/)

its not that theres something wrong with http or whatever its that the protocol is more complex than it strictly needs to be for many uses. Gemini is very clear about their intent to NOT try to be an http replacement.

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

are HTML and Gemini incompatible?

my very first paid programming was hacking on the gopher server source in a very Canadian project, and I couldn’t figure out how to keep it from crashing so I removed some free()s from my code and we just restarted it every night

when we moved our service to http/web stuff I wrote a CGI thing that just connected to the Gopher server behind it and served that content, because I wasn’t convinced that the web thing was going to work (and I didn’t want to have to make new upload stuff for our content managers). hyper-g honestly seemed better to me!

rotor
Jun 11, 2001

classic case of pineapple derangement syndrome

Subjunctive posted:

are HTML and Gemini incompatible?

html and gemini are incompatible in the same way that ssh and html are incompatible, ie they are different things with different intended uses

fart simpson
Jul 2, 2005

DEATH TO AMERICA
:xickos:

rotor posted:

html and gemini are incompatible in the same way that ssh and html are incompatible, ie they are different things with different intended uses

u can tunnel all sorts of junk over ssh tho

fart simpson
Jul 2, 2005

DEATH TO AMERICA
:xickos:

fun fact: tunneling all your web traffic over ssh was one of the og ways around the great firewall

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

rotor posted:

html and gemini are incompatible in the same way that ssh and html are incompatible, ie they are different things with different intended uses

wait, I thought Gemini was for distributing hypertext documents — that seems like something that could
interact meaningfully with HTML

the protocol specification is separate from the gemtext document specification, even

I would imagine that you could distribute gemtext over HTTP, similarly, because they are compatible?

maybe you mean something unusual by “incompatible”? like a program is compatible with an operating system, but they are different things

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

oh man transient client certificates this is good stuff

rotor
Jun 11, 2001

classic case of pineapple derangement syndrome

Subjunctive posted:

maybe you mean something unusual by “incompatible”? like a program is compatible with an operating system, but they are different things

yeah, i just thought the question you posed was oddly phrased. HTML and Gemini are different things. You CAN use Gemini to transfer html but theres not a lot of inspection of the files it transfers, and for reasons that faq should make clear.

rotor
Jun 11, 2001

classic case of pineapple derangement syndrome
gopher was a lot more like anonymous ftp than it was like http

Adbot
ADBOT LOVES YOU

BattleMaster
Aug 14, 2000

*taps the sign that says OSI Model*

I figured most transfer protocols have no awareness of what's being transferred. HTTP and Gopher at least at their core take file paths and send you the contents. Gopher is even far more specifically that since it spares the pleasantries and doesn't even tell you file sizes or protocol versions. You give it a path and it gives you the file with nothing else going on.

HTTP sends MIME types and Gopher clients are aware of what kind of selector type you had just picked from the menu, but from what I've seen they mostly just send you the data and let you figure out what do do with it.

(Gopher has that kind of boneheaded thing where they expect you to escape periods that follow CRLF sequences so the client doesn't get confused about what the end of the file is, but like, maybe just keep reading until you're fed up or the server indicates EOF???)

Edit: Although maybe the question was really if clients have awareness of those things? Like Gopher clients aren't going to necessarily know how to render HTML even if they can download it just fine, while web clients don't have Gopher support or even the ability to render the menus anymore.

But even if Gopher has a specific menu file format in its spec, the network protocol has no real awareness of that and sends it like any other file.

BattleMaster fucked around with this message at 18:25 on May 4, 2024

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply