Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

Hopefully quick Python installation question: I've installed 3.7.3 onto a Windows server, confirmed the path is set, and restarted it. I can now execute commands from command line, PowerShell, etc. locally just fine, but the same commands produce a "Can't find a default Python" error if I attempt to execute them as a remote batch job.

The remote user has administrative rights, any ideas of what I might be missing?

# ¿ May 29, 2019 04:53

Adbot: ADBOT LOVES YOU

# ¿ May 2, 2024 08:41

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

Proteus Jones posted:

The only thing I can think of off the top of my head the $PATH for python is not being passed to the remote user an environment setting.

What happens when you execute with the full pathname to python in the script?

If I run it directly it works fine. If I try specifying the version, the py launcher helpfully informs me that it can't detect Python being installed.

Test script (PyTest.py just runs print("test")):

code:

& 'C:\Program Files (x86)\Python3.7.3\python' "\\filepath\PyTest.py"
py -3.7 "\\filepath\PyTest.py"

Result:

code:

test
Installed Pythons found by C:\Windows\py.exe Launcher for Windows
No Installed Pythons Found!

Requested Python version (3.7) not installed, use -0 for available pythons
Python 3.7 not found!

Again, run locally just "py" works, but "py -3.7" works as well. I can just force it to do the full path every time, but that seems really silly and denies me the ability to execute code from my automation platform directly.

# ¿ May 29, 2019 13:51

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

Whats the �best� way using boto3 to read S3 content into a stream that can be used in in a single POST request? I�d prefer to stick with Lambda �native� libraries if possible but it�s not critical.

# ¿ Feb 15, 2022 13:06

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

CarForumPoster posted:

It'd be like 3 lines of code if you use boto3+requests.

Get the file object as bytes via boto3, transmit file object using requests.

To clarify, I�m looking to post like 10-20GB of data so I�m looking to stream it. Loading the whole file into memory isn�t going to be feasible in Lambda.

# ¿ Feb 15, 2022 14:40

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

Bad Munki posted:

If you�re using lambda (and, say, api gateway) and hoping to stream that content back to a client, it won�t work the way you want, lambda returns its response en bloc, unless they�ve changed something fundamental about it since I last had to deal with this.

You said in a single POST request, so I understand you�re the one making the request as a client, not fielding it as a server, but just in case, be aware of that limitation.

None of what I said actually pertains to streaming a file from S3, though. But you said �lambda� and �stream� and it caught my attention.

I appreciate you checking. I'm invoking the an AWS Lambda function through Step Functions and using it to push data via API (currently using urllib3 since it's baked into the image and simple enough). The Lambda return value is going to be discarded.

# ¿ Feb 15, 2022 14:55

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

Anyone have any experience with Paramiko throwing an error during a rename? I can connect to the SFTP server and download a file from that remote host, but asking it to move the remote file (into remote archive path) results in an "[Errno 2] File not found". I can use the same client to lstat both the original filepath and the destination directory filepath and both return the appropriate information. I have full admin rights with the user on the SFTP-side.

This is using an SFTP client object that's created using the .from_transport() method. The other commands I'm issuing with it (list_dir, file) work fine. There's not really much to it:

Python code:

sftp.rename(orig_path, archive_path)

I tried using the .posix_rename() method instead, but I get a "not supported" error which tracks since it's a Windows host. I've dug around on a number of sites and I can't seem to find too many posts about it. Any help would be appreciated.

:confused:

# ¿ Mar 24, 2022 17:15

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

It�s Globalscape EFT, a relatively robust but persnickety hosting tool.

I think it�s probably just going to be a no-go and I�ll just redo it from the SFTP-end and manipulate things at the OS. I�m genuinely curious is there�s anything that could be done to get it working but I�m not interested enough not to take the easy fix.

# ¿ Mar 24, 2022 19:28

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

I�m looking to replicate OpenSSL aes-256-cbc decryption of a .zip file using an RSA .pem public key file. We�ve got a script that just runs the command line executable and that works fine, but they want to port it to a AWS Lambda function. I�ve looked over a variety of pyopenssl and pycryptodome examples and I just cannot seem to dig up a working example that properly converts the .pem format to AES with the right results. Has anyone had any experience they can share on this?

# ¿ Feb 23, 2023 02:29

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

QuarkJets posted:

I don't, and this is an unsatisfying answer, but what if you just deploy a container image that runs that script?

It�s not out of the realm of possibility. I�ve managed to convert all their other processes to Python functions or classes so at least as a learning exercise I�d like to make it happen, but I�m not above taking the easy way out.

# ¿ Feb 23, 2023 03:04

Adbot: ADBOT LOVES YOU

# ¿ May 2, 2024 08:41

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

Cyril Sneer posted:

Fun little learning project I want to do but need some direction. I want to extract the all the video transcripts from a particular youtube channel and make them both keyword and semantically searchable, returning the relevant video timestamps.

I've got the scraping/extraction part working. Each video transcript is returned as a list of dictionaries, where each dictionary contains the timestamp and a (roughly) sentence-worth of text:
code:
    {
    'text': 'replace the whole thing anyways right so',
     'start': 1331.08,
     'duration': 4.28
    }
I don't really know how YT breaks up the text, but I don't think it really matters. Anyway, I obviously don't want to re-extract the transcripts every time so I need to store everything in some kind of database -- and in manner amenable to reasonably speedy keyword searching. If we call this checkpoint 1, I don't have a good sense of what this solution would look like.

Next, I want to make the corpus of text (is that the right term?) semantically searchable. This part is even foggier. Do I train my own LLM from scratch? Do some kind of transfer learning thing (i.e., take existing model and provide my text as additional training data?) Can I just point chatGPT at it (lol)?

I want to eventually wrap it in a web UI, but I can handle that part. Thanks goons! This will be a neat project.

This sounds like a good use-case for a vectored database and retrieval-augmented generation (RAG) and/or semantic search. You can use your dialog text as the target material and the rest as metadata you can retrieve on match. Theres a number of free options for database, including local ChromaDB instances (which use SQLite) or free-tier Pinecone.io which has good library support and a decent web UI.

# ¿ Apr 17, 2024 11:49

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python