|
A simple way to see if your code is working is to just write another Python script that reads stdin and then writes it to a file, and then run that script with Popen. If that works, then the issue isn't with your code. Once you get that working, then replace your script with the Terraria executable. Is the server actually running in the way that you expect? I believe this: Python code:
Python code:
|
# ? Jun 15, 2020 07:39 |
|
|
# ? Jun 5, 2024 04:46 |
|
QuarkJets posted:And I don't think it's necessary to launch a shell to launch your process (e.g. you may be able to remove shell=True), but I could be wrong about that. This is correct, as quoted from the Python docs: quote:On Windows with shell=True, the COMSPEC environment variable specifies the default shell. The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable. I assume TerrariaServer.exe runs in the console, I think most game servers run in the console but my last experience hosting was like Minecraft infdev days.
|
# ? Jun 15, 2020 15:27 |
QuarkJets posted:Is the server actually running in the way that you expect? I believe this: code:
quote:And I don't think it's necessary to launch a shell to launch your process (e.g. you may be able to remove shell=True), but I could be wrong about that. Protocol7 posted:I assume TerrariaServer.exe runs in the console, I think most game servers run in the console but my last experience hosting was like Minecraft infdev days. I'm going to try the suggestion of having this thing pipe its IO through a different interactive command. Or better yet, I'll try it on my mac. I wasn't doing it there initially because reasons, but I suspect Windows is jerking me around a bit here so this may change the path of least resistance for the ultimate goal. Bad Munki fucked around with this message at 16:04 on Jun 15, 2020 |
|
# ? Jun 15, 2020 16:02 |
It's a bit happier in OS X. Still had to use the cp437 encoding to fire up the server, I'm assuming that's because the thing was built for windows and it's kind of kludged into other platforms. It's not QUITE there, though. I can run the thing, I can type commands in and they get forwarded to the server console and it runs them. But if I try to programmatically send a command, it looks like it gets filled into the process' stdin buffer, but not actually fired off in spite of a terminating \n (tried \r\n as well) and a flush. But I know the commands are getting in there, because if I sit there at the console and manually hit enter occasionally, it'll run whatever commands my script sent over. It's an improvement over the windows context, at least, and it appears I'm probably just missing some character or statement to trigger the console to say "end of command" and process whatever's in the buffer. Hmm. e: Yeah, it's the darnedest thing. If I sit here with my return key mashed, the whole thing functions exactly as intended, albeit with a few blank lines from the server console because I'm constantly entering blank commands, with the occasional actual command output interspersed as my proxy sends the command but is unable to execute it. The fix here is, obviously, to leave a paperweight on my return key indefinitely. Bad Munki fucked around with this message at 17:23 on Jun 15, 2020 |
|
# ? Jun 15, 2020 17:04 |
|
Bad Munki posted:e: Yeah, it's the darnedest thing. If I sit here with my return key mashed, the whole thing functions exactly as intended, albeit with a few blank lines from the server console because I'm constantly entering blank commands, with the occasional actual command output interspersed as my proxy sends the command but is unable to execute it. The fix here is, obviously, to leave a paperweight on my return key indefinitely. I think you need to set bufsize=1 and universal_newlines=True. edit: Although maybe not? I would have thought a manual flush() would also have done the trick.
|
# ? Jun 15, 2020 18:27 |
|
This poo poo is why I ended giving up and just importing pexpect for interactive PTY sessions (note spawnu() doesn't work in Windows version of pexpect since no PTY support) (it may work if launched from bash w/WSL)
|
# ? Jun 15, 2020 18:46 |
Proteus Jones posted:This poo poo is why I ended giving up and just importing pexpect for interactive PTY sessions (note spawnu() doesn't work in Windows version of pexpect since no PTY support) (it may work if launched from bash w/WSL) Well when you're right you're right. That simplified getting the thing launched and condensed the whole script down to almost nothing. The server still isn't getting my commands, but it feels like a better place to be working from. code:
So it feels closer, but as I said, the server still isn't getting commands. I just get an eventual timeout, and also a little bit of garbage that ends up in my actual command shell command buffer somehow? code:
Bad Munki fucked around with this message at 19:44 on Jun 15, 2020 |
|
# ? Jun 15, 2020 19:24 |
|
in regards tocode:
I'll see if I can dig up my old hacky scripts and see how I dealt with it. I'm pretty sure I had to make some kind of accommodation due to some lovely links to some of the devices I was touching with the script.
|
# ? Jun 15, 2020 19:36 |
Proteus Jones posted:in regards to Awesome, thanks. I also altered it to use child.logfile = sys.stdout instead of my kludgey sys.stdout.write(child.readline()) bit, which suppressed the garbage in the stream, but also suppressed the entire output of the server, so I must be misunderstanding something there still. But at least it indicates the garbage is coming from the server somehow, which I'm more willing to tolerate. Also, it lets me see my commands I'm programmatically sending, so I at least know an attempt is being made to send them. Incidentally, switching to child.logfile = sys.stdout, while not successfully displaying the process output, is preventing the timeout. Things may be working and I'm just not seeing the server output, I'll do some investigation. My script output now looks something like this: code:
e: It's not, the commands my script is sending to the server aren't making it. :/ Another edit: I simplified the script even further for debugging. It's just running zsh and trying to send the date command once a second. The output is baffling: code:
code:
Bad Munki fucked around with this message at 21:56 on Jun 15, 2020 |
|
# ? Jun 15, 2020 19:46 |
|
Bad Munki, all bets are off as soon as you start doing this kind of thing with pipes. Everything indicates this kind of thing should work but it really depends heavily on the application you're wrapping. It also varies by OS, which I think you discovered. It's fairly easy to get something to, say, just pipe everything out to a file or the screen, but once you get into actual pipe interaction, there be dragons. So I can't really say what could be wrong and I imagine you won't get anything straightforward, but some jiggering of this or that thing will probably do it eventually. If there's one area where my spidey sense is going off, it's that you're reading the pipe and just slamming it through without examining. Any particular read could be giving you whatever-what-the-heck partial line of text. Last I saw that was with Python 2.7 and maybe Python 3 grew up, but I don't want to take bets on that. The server stdin might get that input in a way that thinks it got the full input and proceed to eat it. So if you're monitoring that during debug, you will want to see what each read operation is actually getting you. The second thing is that newlines might be getting chewed up so the stuff is making it and never actually getting submitted. Now for just poking at poo poo, I think I saw you were juggling cp437 encoding. I remember at one point using that too for a lot of wrapping but still had all kinds of invalid characters come out of nowhere and screw everything up. This was coming from pretty old, US-based software, so I didn't have any reason to expect a whackjob character. I think I had to eventually retreat to just using utf-8 and settings errors='replace'. The process you're wrapping might be expecting something else from stdin too, just to make things annoying. Re: shell=True: That's generally used if you want to run shell commands like 'dir' that aren't actually applications. Also, the "correct" way to use the command line with shell=True is to pass in the command as a single string instead of a list; use a list for shell=False. Also, I think that's actually bullshit since I've seen it work with a list. I had a bunch of code that was cargo-cult using shell=True for launching applications that I ultimately figured out weren't running shell commands.
|
# ? Jun 15, 2020 21:58 |
I’ve simplified, and am trying a more basic case, dropped any extras, including the game itself, but yeah, if this works, it looks like it’s going to be after something inexplicable and a lot of fiddling. Undoubtedly if I do find the solution, I’ll be certain I’ve tried it before. Uhhhh, I fixed it, it works now. Hilarious. This version did not work, has weird buffering of IO leading to strange pauses and increasingly stacked ignored commands: code:
code:
Bad Munki fucked around with this message at 23:15 on Jun 15, 2020 |
|
# ? Jun 15, 2020 23:02 |
|
Is the only difference the spacing you are expecting with the % character?
|
# ? Jun 16, 2020 03:57 |
Yeah, wrapping the % in a space on either side makes that example work as expected. Why.
|
|
# ? Jun 16, 2020 04:00 |
|
Bad Munki posted:Yeah, wrapping the % in a space on either side makes that example work as expected. Why. I'm not sure if you need the leftmost space, but the right-most is needed since the cursor appears one space past the symbol in most shells and CLIs. I remember it being an issue with the password prompt for the old Cisco Catalysts I was touching. I usually had to define that expect statement as code:
Proteus Jones fucked around with this message at 05:43 on Jun 16, 2020 |
# ? Jun 16, 2020 05:39 |
|
paramiko is indeed sweet
|
# ? Jun 16, 2020 09:29 |
Proteus Jones posted:I'm not sure if you need the leftmost space, but the right-most is needed since the cursor appears one space past the symbol in most shells and CLIs. I remember it being an issue with the password prompt for the old Cisco Catalysts I was touching. I usually had to define that expect statement as Maybe I misread the docs, but I didn't think you had to precisely specify the last character of whatever you're looking for, it'd just match wherever and once it saw the expected pattern, the expect() would stop blocking and the script would move on. In which case, it should work with or without the spaces. If that's not the case, though, that might explain some of my earlier issues. The terraria console doesn't have a normal prompt. You issue a command, it puts the results of that command on a line prefixed with a : character, and then moves the cursor to the next line, and leaves it entirely blank. I guess one could expect('\n') in that case, I don't know how it handles the IO stream internally.
|
|
# ? Jun 16, 2020 16:15 |
|
QuarkJets posted:paramiko is indeed sweet
|
# ? Jun 16, 2020 19:21 |
|
So despite instructions to deliver everything as .jpgs and/or .pngs I ended up getting a ton of images for a project that are .dng format. They have GPS location I am trying to use available in the Windows info dialog, but it's not embedded into the image as EXIF. I can pull EXIF data out of .jpgs with PIL but it barfs on .dngs. Is there some way to pull this info out with Python?
|
# ? Jun 16, 2020 23:49 |
|
Protocol7 posted:So despite instructions to deliver everything as .jpgs and/or .pngs I ended up getting a ton of images for a project that are .dng format. A quick google indicates there may be some command line tools that convert DNG to TIFF or others. Can you convert on the command line and then extract with PIL?
|
# ? Jun 17, 2020 00:19 |
|
I'd never heard of the dOng pic format before
|
# ? Jun 17, 2020 00:33 |
|
CarForumPoster posted:A quick google indicates there may be some command line tools that convert DNG to TIFF or others. Can you convert on the command line and then extract with PIL? I could, I guess if it’s some obscure or brand new format that’s probably my only option.
|
# ? Jun 17, 2020 00:57 |
|
Protocol7 posted:I could, I guess if it’s some obscure or brand new format that’s probably my only option. Is this like a production, customer facing app or just something you want for you? A little more googling indicates that GIMP can: read DNGs, read file metadata, have python plugins made for it and call those plugins from the command line. Maybe try and see if GIMP's GUI can open and view the metadata and if so try that?
|
# ? Jun 17, 2020 02:12 |
|
CarForumPoster posted:Is this like a production, customer facing app or just something you want for you? Maybe ImageMagick? It supports digital negatives and I would imagine there's at least one library out there that integrates it. At the worst invoke the command directly in Python. GIMP seems like a big install to just rip EXIF out of files. Another thought is that from my (very cursory) reading, DNF is really just a TIFF variation. So maybe there's python stuff to manipulate/import TIFF metadata. Since you're not actually interacting with the pixels themselves and just want EXIF it shouldn't matter that it's TIFF-but-really-a-negative. Proteus Jones fucked around with this message at 03:05 on Jun 17, 2020 |
# ? Jun 17, 2020 03:02 |
|
DNG is a digital negative file from lightroom I think
|
# ? Jun 17, 2020 04:24 |
|
taqueso posted:DNG is a digital negative file from lightroom I think
|
# ? Jun 17, 2020 12:16 |
|
CarForumPoster posted:Is this like a production, customer facing app or just something you want for you? The files will eventually get uploaded so it’s probably best to convert them to a more web friendly format like JPG anyway. I did some reading and the imagemagick tool is probably the best for a conversion like this, so I’ll be giving that a shot to convert a batch. The python part of it is just a simple script to run a query for a record via lat/long on a 3rd party API and upload the photo to that record based on the results of that query. So maybe this question would have been better posed in the general thread. Either way I’m on a better path now than I was before.
|
# ? Jun 17, 2020 14:21 |
|
CarForumPoster posted:I just did this instead: A little late but the advantage of pandas is that you can do a whole bunch of assignments at once. You shouldn't have to loop here. I suspect this will work: Python code:
|
# ? Jun 18, 2020 18:39 |
|
Proteus Jones posted:GIMP seems like a big install to just rip EXIF out of files. This is a pet peeve of mine with developers of proof of concepts. Memory/compute cycles are cheap.* Programmer time and getting customer feedback is not. Spending time making elegant code means you wasted a bunch of time that could be validating that users actually want this feature. Obviously this doesnt apply to production critical stuff or existing UI where people already expect a certain performance level. *in most settings SurgicalOntologist posted:A little late but the advantage of pandas is that you can do a whole bunch of assignments at once. You shouldn't have to loop here. I suspect this will work: Yea great suggestion!
|
# ? Jun 18, 2020 21:00 |
|
I would poo poo all over somebody for delivering a poc using GIMP instead of something like ImageMagick that is actually made to be used programmatically.
|
# ? Jun 18, 2020 21:42 |
|
necrotic posted:I would poo poo all over somebody for delivering a poc using GIMP instead of something like ImageMagick that is actually made to be used programmatically. To be fair, gimp's batch mode is designed for programmatic use. But I agree that it is pretty horrible to load a massive application just to convert an image. CarForumPoster posted:This is a pet peeve of mine with developers of proof of concepts. But there's also knowing how to use the right tool for a job, which is what this is. Like "and then to generate a random number the code runs a VirtualBox image that opens a connection to random.org" should surely not be met with "well compute cycles are cheap after all" QuarkJets fucked around with this message at 23:27 on Jun 18, 2020 |
# ? Jun 18, 2020 23:25 |
|
The Cheap assumption isn't as universal as you may think - eg anything not running on a general computer, that requires real-time responsiveness, or runs on a computer bogged down by enterprise security software. It's not a bad assumption for image editing software, but is common to hear generally, where it's not true.
|
# ? Jun 18, 2020 23:55 |
|
QuarkJets posted:But there's also knowing how to use the right tool for a job, which is what this is. Like "and then to generate a random number the code runs a VirtualBox image that opens a connection to random.org" should surely not be met with "well compute cycles are cheap after all" I feel like my objection to that approach is more often going to be that it adds a lot of unneeded complexity rather than that it's computationally intensive which is also likely to be my objection to using gimp instead of something better suited for the job.
|
# ? Jun 19, 2020 14:28 |
|
If its a prototype and I'm asking them about it for something like an interview, I'd want to hear why they picked GIMP. IMO it'd be great if the answer is "I had just used it as a command line tool a couple weeks earlier so I knew exactly how to get it do what I needed." Then I can ask what they would do if it became a real thing. A bad answer might be "It's the only way I could figure out to do it" or "I dunno, its what stack overflow said to do"
|
# ? Jun 19, 2020 15:04 |
|
I wouldn't rib a dev over something like that over a POC, but I'd expect that if the POC gets promoted into a real feature that the libraries they use are more carefully selected.
|
# ? Jun 19, 2020 15:38 |
|
Dominoes posted:The Cheap assumption isn't as universal as you may think - eg anything not running on a general computer, that requires real-time responsiveness, or runs on a computer bogged down by enterprise security software. It's not a bad assumption for image editing software, but is common to hear generally, where it's not true. Yea completely agree that requirements should drive method selection. Its been my experience so far though that developers, particularly younger ones, would much rather write a bunch of (often buggy) code than use something off the shelf. Their reason is often something about "efficiency/speed/etc." when that isn't a requirement of the task at hand.
|
# ? Jun 19, 2020 23:56 |
|
Protocol7 posted:I wouldn't rib a dev over something like that over a POC, but I'd expect that if the POC gets promoted into a real feature that the libraries they use are more carefully selected. Expect to be disappointed lol
|
# ? Jun 20, 2020 06:22 |
|
QuarkJets posted:Expect to be disappointed lol Sorry about your lovely coworkers.
|
# ? Jun 20, 2020 13:17 |
|
Protocol7 posted:Sorry about your lovely coworkers. Yeah the entirety of the software development world is pretty lovely
|
# ? Jun 20, 2020 14:26 |
|
QuarkJets posted:Yeah the entirety of the software development world is pretty lovely Also lazy as gently caress.
|
# ? Jun 20, 2020 14:27 |
|
|
# ? Jun 5, 2024 04:46 |
|
Proteus Jones posted:Also lazy as gently caress. Simple is better than complex. If the implementation is easy to explain, it may be a good idea.
|
# ? Jun 20, 2020 16:20 |