Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

Foxfire_: Nov 8, 2010

That doesn't actually kill the process, it just stops waiting for the result:

Python code:

import concurrent.futures
import time

def forever():
    while True:
        time.sleep(1.0)
        print("I'm not dead yet")

def main():
    with concurrent.futures.ProcessPoolExecutor() as executor:
        try:
            future = executor.submit(forever)
            time.sleep(5.0)
            future.cancel()
            return 0

        except concurrent.futures._base.TimeoutError:
            raise RuntimeError("Timeout")

        finally:
            print("Shutting down ProcessPool...")
            executor.shutdown(wait=False, cancel_futures=True)


if __name__ == '__main__':
    print(main())

I get

code:

(py392) D:\trash>python stuff.py
I'm not dead yet
I'm not dead yet
I'm not dead yet
I'm not dead yet
Shutting down ProcessPool...
0
I'm not dead yet
I'm not dead yet
I'm not dead yet
I'm not dead yet (repeats forever)

mr_package posted:

I was also surprised that you basically cannot easily kill threads/processes that you launch. There are third-party libraries and what appear-- to me-- to be very hack-y approaches to dealing with this to be found on SO. Perhaps they are fine, perhaps this is just the way it is done. This is my first look at concurrency, do other languages handle this pretty much the same way or is it Python-specific? I'm thinking for example if you were writing something in go or rust or C# or whatever and you wanted to terminate a process early, is it easier?

You may never safely abruptly kill a thread in any language. If you want the ability to cancel things, you need to make your code voluntarily die. The Win32 syscall that does it (TerminateThread) for example is full of nasty warnings that basically boil down into "Never ever call this". The pthreads one (pthread_kill) has less warnings in its docs, but isn't any safer to use.

It is possible to safely abruptly kill a process if you have appropriate permissions, but it's not something I would expect a processing pool library to make easy. Again, your code should have a way to gracefully cancel (returning the worker to the pool), not forcefully rip down a piece of the pool infrastructure and hope the pool implementation heals.

# ? Jun 11, 2021 03:43

Adbot: ADBOT LOVES YOU

# ? May 13, 2024 21:44

mr_package: Jun 13, 2000

What does the correct way to do this look like? I have a piece of code I know might take too long. How do I set up the call to it so that it terminates in an appropriate time? This is not a standard thing?

Or, is it that the core issue is socket.gethostbyname() has a 5s timeout and I need to respect that? In other words, shoehorning control of that process into a MP library is the wrong approach: I should not use socket.gethostbyname() at all if the timeout behavior is wrong for my use case.. For example, I could write my own (what could go wrong?) or use another library such as https://www.dnspython.org/ that has a timeout and kills itself rather than my trying to kill it if it takes too long. Is that the way this type of thing should be done?

As an aside, does anyone know offhand what happens when you call subprocess.run() with a timeout? Does the process continue to run in the background and then exit normally however long it takes? I'm wondering if the caller just ignores it and moves on or if it actually communicates to the process that it should quit/cancel (presumably the process would be responsible for then handling this in a mostly graceful way).

# ? Jun 11, 2021 06:42

Foxfire_: Nov 8, 2010

mr_package posted:

What does the correct way to do this look like? I have a piece of code I know might take too long. How do I set up the call to it so that it terminates in an appropriate time? This is not a standard thing?

Or, is it that the core issue is socket.gethostbyname() has a 5s timeout and I need to respect that? In other words, shoehorning control of that process into a MP library is the wrong approach: I should not use socket.gethostbyname() at all if the timeout behavior is wrong for my use case.. For example, I could write my own (what could go wrong?) or use another library such as https://www.dnspython.org/ that has a timeout and kills itself rather than my trying to kill it if it takes too long. Is that the way this type of thing should be done?

You can't kill it if it's in the same process as you because it is potentially using global stuff you would also like to use (e.g. if it's in the middle of allocating memory, it may be holding a mutex for the C runtime heap allocator). Ideally, functions would give you a way to tell it to cancel itself that will make it nicely exit

For lots of stuff that wasn't ever intended to be used asynchronously, there won't be a way to do that. gethostbyname() doesn't have a way to cancel it. What you can do is make the code that launched it not wait anymore & make it so that once the slow call does return, your code ignores the result and just exits:

Python code:

def do_the_thing():
  address = socket.gethostbyname("dick.butt")
  if am_i_canceled():
    return
  
  do_stuff_with_address(address)

If your calling code does something to make am_i_canceled() return true, whenever the gethostbyname call does return, it will exit then. It's not really 'canceled' in terms of an abnormal execution path, you're just adding a signaling mechanism and execution path that exits earlier than it would otherwise

mr_package posted:

As an aside, does anyone know offhand what happens when you call subprocess.run() with a timeout? Does the process continue to run in the background and then exit normally however long it takes? I'm wondering if the caller just ignores it and moves on or if it actually communicates to the process that it should quit/cancel (presumably the process would be responsible for then handling this in a mostly graceful way).

You can look at the CPython implementation to see what it does (https://github.com/python/cpython/blob/main/Lib/subprocess.py). On timeout, it calls the python function kill() on the python process object. The documentation for that says that it sends a SIGKILL on Linux/Mac and calls TerminateProcess() on Windows. Neither is ignorable, the operating system will abruptly kill that victim process (which is safe to do* because it doesn't share any state with the parent process. (*unless you specifically set up an inter-process communication method, in which case it would no longer be safe))

# ? Jun 12, 2021 01:38

Data Graham: Dec 28, 2009; 📈📊🍪😋

Hey yo. What's the least stupid way to do the following:

* Define a "templated" arithmetic function in a string, for example "0.5 * {width}"
* Store that string in a database (i.e. a Django CharField)
* At rendering time, eval the string against a set of kwargs and spit out a result

I know I can just like eval(templated_string.format(**kwargs)), but that seems dangerous even if I own the data flow end-to-end. Is there a good way to do this that avoids eval()?

# ? Jun 13, 2021 00:49

QuarkJets: Sep 8, 2008

I think eval() is probably the least stupid way to do it, you just need to do some text validation on the input to ensure that you're seeing arithmetic and not something else. Any non-arithmetic characters should raise an exception prior to running with eval()

# ? Jun 13, 2021 01:05

breaks: May 12, 2001

The old Armin Ronacher article about security concerns with user provided format strings might also be worth a read. On mobile but should be easy to Google.

# ? Jun 13, 2021 02:19

SurgicalOntologist: Jun 17, 2004

It's probably overkill and I have no idea what it offers for serializing expressions, but it could be interesting to use the symbolic computation library SymPy to represent variables and expressions. I think it can parse strings so you wouldn't have to worry about that. You probably don't need much math stuff but you can evaluate expressions, and maybe some other features like simplification or equation display would be appealing.

# ? Jun 13, 2021 08:20

Walh Hara: May 11, 2012

Data Graham posted:

Hey yo. What's the least stupid way to do the following:

* Define a "templated" arithmetic function in a string, for example "0.5 * {width}"
* Store that string in a database (i.e. a Django CharField)
* At rendering time, eval the string against a set of kwargs and spit out a result

I know I can just like eval(templated_string.format(**kwargs)), but that seems dangerous even if I own the data flow end-to-end. Is there a good way to do this that avoids eval()?

Probably a stupid idea: what about storing the arithmetic function as a serialized object in the database?

So instead of a storing the string "0.5 * {width}", you could serialize the function "lambda width: 0.5 * float(width)" into a string and store that. Afterwards you can retrieve the string, deserialize it into a function and call it.

See this SO for ways to do that:
https://stackoverflow.com/questions/1253528/is-there-an-easy-way-to-pickle-a-python-function-or-otherwise-serialize-its-cod

Note: I don't think this is actually that much safer than your version if you're worried about people modifying the database (on the contrary). But it should be more safe if you're worried about code injection via the width parameter.

edit: actually, perhaps storing "lambda width: 0.5 * float(width)" instead of "0.5 * {width}" might be worth considering as well? Obviously more complicated, but python will complain if "width" is not parseable to a float.

Walh Hara fucked around with this message at 09:37 on Jun 13, 2021

# ? Jun 13, 2021 09:24

Data Graham: Dec 28, 2009; 📈📊🍪😋

Good feedback, thanks all. I guess I'll just preface the eval() with some regex checking, since these are pretty simple expressions I'm working with. Maybe a restrictive length limit too in case someone is somehow able to fit "import User" and "User.objects.all().delete()" into it lol

I mainly wanted to see if there was a commonly-used best-practices approach for this sort of thing that I didn't know about.

# ? Jun 13, 2021 12:34

Phobeste: Apr 9, 2006; never, like, count out Touchdown Tom, man

I mean unfortunately I think the sort of real answer is "if it matters, don't do that". Like, if this is a toy project, whatever, string.format() is probably fine. But if it's a real project that's, for instance, on the internet, you probably don't want to use strings for it at all. You want to store a computation tree, like an AST (and maybe literally an ast.AST instance that you poke the user-supplied operands into), after parsing and sanitizing user input at the point of entry, both for security reasons and for error locality reasons.

# ? Jun 13, 2021 13:26

SirPablo: May 1, 2004; Pillbug

I'm having a hell of a time trying to figure out how to parse this into a dictionary.

https://forecast.weather.gov/MapClick.php?lat=41.4273&lon=-90.0224&unit=0&lg=english&FcstType=dwml

# ? Jun 17, 2021 00:31

sleppy: Dec 25, 2008

Why not use their API?
https://api.weather.gov/points/41.4273,-90.0224
https://weather-gov.github.io/api/general-faqs

# ? Jun 17, 2021 01:58

SirPablo: May 1, 2004; Pillbug

sleppy posted:

Why not use their API?
https://api.weather.gov/points/41.4273,-90.0224
https://weather-gov.github.io/api/general-faqs

I'd love to butt unfortunately it has severe caching issues that aren't going to be resolved any time soon. Kind of a big deal for time sensitive information.

https://github.com/weather-gov/api/discussions/224

# ? Jun 17, 2021 02:54

Deadite: Aug 30, 2003; A fat guy, a watermelon, and a stack of magazines?
Family.

I'm having an issue trying to add a matplotlib graph to a tkinter GUI where the legend to the graph is getting cut off if I move it to below the chart.

Does anyone know a way to display the legend when it is outside of the chart? FigureCanvasTkAgg doesn't have a height argument so I can't just stretch the viewable area. It looks like it is some kind of automatic resizing problem that I can't figure out a way around.

Here's the test code:

Python code:

import tkinter as tk
from pandas import DataFrame
import matplotlib.pyplot as plt
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg

data1 = {'Country': ['US','CA','GER','UK','FR'],
         'GDP_Per_Capita': [45000,42000,52000,49000,47000]
        }
df1 = DataFrame(data1,columns=['Country','GDP_Per_Capita'])

root= tk.Tk() 
  
figure1 = plt.Figure(figsize=(6,5), dpi=100)
ax1 = figure1.add_subplot(111)
bar1 = FigureCanvasTkAgg(figure1, root)
bar1.get_tk_widget().pack(side=tk.LEFT, fill=tk.BOTH)
df1 = df1[['Country','GDP_Per_Capita']].groupby('Country').sum()
df1.plot(kind='bar', legend=True, ax=ax1)
ax1.legend(bbox_to_anchor=(0.7, -0.12)) 
ax1.set_title('Country Vs. GDP Per Capita')

root.mainloop()

# ? Jun 17, 2021 14:42

Data Graham: Dec 28, 2009; 📈📊🍪😋

SirPablo posted:

I'd love to butt unfortunately it has severe caching issues that aren't going to be resolved any time soon. Kind of a big deal for time sensitive information.

https://github.com/weather-gov/api/discussions/224

Gross.

I would look at https://docs.python.org/3/library/xml.etree.elementtree.html
and adapt this https://stackoverflow.com/questions/2148119/how-to-convert-an-xml-string-to-a-dictionary

# ? Jun 17, 2021 14:49

SirPablo: May 1, 2004; Pillbug

Thanks will give that a try.

# ? Jun 17, 2021 14:52

OnceIWasAnOstrich: Jul 22, 2006

Deadite posted:

I'm having an issue trying to add a matplotlib graph to a tkinter GUI where the legend to the graph is getting cut off if I move it to below the chart.

Does anyone know a way to display the legend when it is outside of the chart? FigureCanvasTkAgg doesn't have a height argument so I can't just stretch the viewable area. It looks like it is some kind of automatic resizing problem that I can't figure out a way around.

I've never had this problem in the context of another GUI but I've definitely run into similar issues with bits of a matplotlib figure getting rendered outside the bounds of an image. It is usually something that a call to tight_layout() or other layout-modifying functions can address.

# ? Jun 17, 2021 14:57

DoctorTristan: Mar 11, 2006; I would look up into your lifeless eyes and wave, like this. Can you and your associates arrange that for me, Mr. Morden?

Have I misunderstood pip�s version specifiers, or is it doing something weird here?

If I run

code:

pip install msal>=1.9.0

I thought this means �install the most suitable version of msal above 1.9.0� ? Instead it downloads four versions of msal and spits out an Error: ResolutionImpossible with the following info:

code:

The conflict is caused by:
msal 1.12.0 depends on cryptography <4 and >= 0.6
msal 1.11.0 depends on cryptography <4 and >= 0.6
msal 1.10.0 depends on cryptography <4 and >= 0.6 
msal 1.9.0 depends on cryptography <4 and >= 0.6

I don�t have the cryptography package installed, so can�t see any conflict here? Unless it�s trying to install all four versions and finding a conflict?

(I originally found this problem with a requirements.txt file that had a bunch of >= dependencies, also this is installing from an Azure artefacts repository that�s mirroring PyPI if that makes any difference)

# ? Jun 17, 2021 15:05

Walh Hara: May 11, 2012

I got myself familiar with xpath once, and now it's the single hammer I use for every single XML parsing problem I encounter. Xml.etree already has a decent support for the syntax.

# ? Jun 17, 2021 15:13

Wallet: Jun 19, 2006

DoctorTristan posted:

I don�t have the cryptography package installed, so can�t see any conflict here? Unless it�s trying to install all four versions and finding a conflict?

(I originally found this problem with a requirements.txt file that had a bunch of >= dependencies, also this is installing from an Azure artefacts repository that�s mirroring PyPI if that makes any difference)

Is it possible that something else in the requirements file has a cryptography dependency? I think pipdeptree will let you know.

# ? Jun 17, 2021 15:22

OnceIWasAnOstrich: Jul 22, 2006

DoctorTristan posted:

Have I misunderstood pip�s version specifiers, or is it doing something weird here?

No, you are using it right. Something in the dependency resolution it is doing seems to be calling for cryptography. I would normally say that something else you have installed or are installing has a conflict with the cryptography version number. What version of Python are you using? Maybe there are no cryptography packages for your Python in that range or the repo/your environment is busted.

You can also try pip installing a version of the package in the appropriate version range and see what it does. Now that I think about it if there was an actual conflicting dependency pip would normally list it in that same output.

OnceIWasAnOstrich fucked around with this message at 15:28 on Jun 17, 2021

# ? Jun 17, 2021 15:23

DoctorTristan: Mar 11, 2006; I would look up into your lifeless eyes and wave, like this. Can you and your associates arrange that for me, Mr. Morden?

OnceIWasAnOstrich posted:

No, you are using it right. Something in the dependency resolution it is doing seems to be calling for cryptography. I would normally say that something else you have installed or are installing has a conflict with the cryptography version number. What version of Python are you using? Maybe there are no cryptography packages for your Python in that range or the repo/your environment is busted.

You can also try pip installing a version of the package in the appropriate version range and see what it does. Now that I think about it if there was an actual conflicting dependency pip would normally list it in that same output.

Thanks.

Should have thought of doing that that myself, but tried it and found that the underlying problem is that there�s no cryptography package on this piece of poo poo azdo mirror of ours, so none of the packages can install.

No idea why pip wouldn�t just say that instead of bullshitting about conflicts, but hey ho.

In conclusion: gently caress computers

# ? Jun 17, 2021 16:16

Hed: Mar 31, 2004; Fun Shoe

Isn�t cryptography the one that switched to linking against rusttls recently?

May not be a big deal depending on your environment but something to keep in mind.

Also, agreed re: computers.

# ? Jun 17, 2021 16:31

Deadite: Aug 30, 2003; A fat guy, a watermelon, and a stack of magazines?
Family.

OnceIWasAnOstrich posted:

I've never had this problem in the context of another GUI but I've definitely run into similar issues with bits of a matplotlib figure getting rendered outside the bounds of an image. It is usually something that a call to tight_layout() or other layout-modifying functions can address.

Thanks, tight_layout is exactly what I was looking for.

# ? Jun 17, 2021 16:49

DoctorTristan: Mar 11, 2006; I would look up into your lifeless eyes and wave, like this. Can you and your associates arrange that for me, Mr. Morden?

Hed posted:

Isn�t cryptography the one that switched to linking against rusttls recently?

May not be a big deal depending on your environment but something to keep in mind.

Also, agreed re: computers.

Possibly true - I don�t know much about it other than that msal depends on it (msal is the Microsoft authentication library - employer uses OAuth all over the shop so I have to use this package a lot)

I also found that if I try to do this with two packages with missing dependencies I can send pip into an infinite loop, which is really stupid.

# ? Jun 18, 2021 10:58

punk rebel ecks: Dec 11, 2010; A shitty post? This calls for a dance of deduction.

nvm

punk rebel ecks fucked around with this message at 02:05 on Jul 11, 2021

# ? Jul 11, 2021 01:31

CarForumPoster: Jun 26, 2013; â¡POWERâ¡

Pandas question:

I am trying to make row wise exponential regression in order to fill in NaN values in a dataframe. Size of data is only 16K rows x 7 cols, so this will work fine but I get a twinge when I iterate over a pandas object in a function I'm going to apply. Whats the right way to accomplish this?

Example data:

code:

columns = [2, 4, 8, 12, 20, 40, 50]
values = [
[NaN, 0.25, 0.25, 0.25, 0.3, 0.3, 0.3],
[2750.0, 4300.00, 6250.00, 11250.00, 17500.0, 26000.0, 31000.0],
[300.0, 750.00, 1150.00, 1500.00, NaN, 6000.0, 6850.0]
]
df = pd.DataFrame(values, columns=columns)

y1 should be the value getting set on the NaNs and grade the column with the NaN. For example the NaN in index 0, col "2" should be approximately 0.25.

I made the below code which returns a list with all the values filled in, note the for loop. This "feels bad" but I can't think of a better way.

code:

from math import exp, isnan

# Pass this a pandas series (.apply)
def get_values(row):
    # If there are any NaNs, fill them
    if row.isnull().values.any():
        new_row = []
        mask = row[~row.isnull()]
        x = np.array(mask.index.values).astype(np.float64)
        y = np.array(mask.values).astype(np.float64)

        # Calculate the regression for the data we have
        val1, val2 = np.polyfit(x, np.log(y), 1, w=np.sqrt(y))

        # Calculate whats needed for the nan values
        for grade, value in row.iteritems():
            if isnan(value):
                new_row.append(
                    exp(val2)*exp(val1*grade)
                )
            else: 
                new_row.append(value)
        return new_row
    else:
        return row.values

# ? Jul 13, 2021 13:53

DoctorTristan: Mar 11, 2006; I would look up into your lifeless eyes and wave, like this. Can you and your associates arrange that for me, Mr. Morden?

Does DataFrame.interpolate() do what you need ?

# ? Jul 13, 2021 14:22

CarForumPoster: Jun 26, 2013; â¡POWERâ¡

DoctorTristan posted:

Does DataFrame.interpolate() do what you need ?

fuckin probably

I didnt see an exponential curve at first glance but I bet one of those will work. At a glance it looks like it wouldnt want to fill backward, but I prob just need to study docs a bit more. This data never dips below 0 and is always exponentially increasing to the right.

I always tell people "someone already probably did it better" when they have a python usecase that seems like it should be common. Didnt follow my own advice.

Thanks!

# ? Jul 13, 2021 14:39

CarForumPoster: Jun 26, 2013; â¡POWERâ¡

DoctorTristan posted:

Does DataFrame.interpolate() do what you need ?

TY again. Hilariously easy and fast solution. Take the log of the df, interpolate linearly across row, take exp of df.

code:

log_df = np.log(df)
log_df = log_df.interpolate(method='linear', axis='columns')
df = np.exp(log_df)

I've lost any math skills I learned in college.

# ? Jul 13, 2021 15:15

Soylent Pudding: Jun 22, 2007; We've got people!

I'm writing my first python app that other people will use instead of just for my own purposes. How do I set things up for users to install the dependencies and run it?

Is it just put in the readme:

code:

Setup:
1. Clone from git repo
2. Set up virtual environment
3. pip install  -r requirements.txt
4. Run app.py --flags

For simplicity sake is their a best practice for automating the install? Like have an install.py that will set up everything if needed and dump them into a terminal in the virtual environment ready to run the app?

# ? Jul 13, 2021 21:52

QuarkJets: Sep 8, 2008

It depends on who you're targeting, you can freeze python applications into executables or just have them run on a webserver if you really want to make it easy. For someone cloning your repo, those instructions are sufficient, publishing to PyPi would be better though

# ? Jul 13, 2021 22:37

former glory: Jul 11, 2011

I'm pretty new to python and I've hit a major stumbling block in trying to test out an old github repo. It has a requirements.txt with a good 7-8 libraries listed with specific versions, and I've inferred that this was developed on python 3.5. I've tried creating a venv with my system 3.7 python and using pip and it dies trying to install one of the libs.

I tried using a conda environment with all sorts of python versions from 3.4-3.9 -- thinking maybe conda is smarter about this than pip -- and it dies on a different package with each version. There doesn't seem to be a way to get all the requirements to install. I've managed to get slightly different versions of libs to install except for matplotlib and pyclipper now, and I'm wondering if there's some sort of standard way of working this out and maybe I'm just making this hard for no reason? I've tried having PyCharm process the requirements.txt with conda and even manually just installing the packages.

Sorry, don't want to get all stackoverflow on the thread, just wondering if there's some obvious thing I should be using for this type of problem. :shrug:

The whole world of pip, conda, miniconda, anaconda has me reaching all over.

# ? Jul 15, 2021 03:01

Phobeste: Apr 9, 2006; never, like, count out Touchdown Tom, man

former glory posted:

I'm pretty new to python and I've hit a major stumbling block in trying to test out an old github repo. It has a requirements.txt with a good 7-8 libraries listed with specific versions, and I've inferred that this was developed on python 3.5. I've tried creating a venv with my system 3.7 python and using pip and it dies trying to install one of the libs.

I tried using a conda environment with all sorts of python versions from 3.4-3.9 -- thinking maybe conda is smarter about this than pip -- and it dies on a different package with each version. There doesn't seem to be a way to get all the requirements to install. I've managed to get slightly different versions of libs to install except for matplotlib and pyclipper now, and I'm wondering if there's some sort of standard way of working this out and maybe I'm just making this hard for no reason? I've tried having PyCharm process the requirements.txt with conda and even manually just installing the packages.

Sorry, don't want to get all stackoverflow on the thread, just wondering if there's some obvious thing I should be using for this type of problem. The whole world of pip, conda, miniconda, anaconda has me reaching all over.

Honestly it kind of sounds to me that time has ended up making that collection of libraries at the versions frozen in the requirements mutually incompatible through transitive dependency, like maybe

- app requires a at version 1 and b at version 2
- a at version 1 requires c at version > 3
- c now has a version 4, which matches a's requirement, but in turn requires, say, b at version 3
- oops you're broken

I don't think there's a specific tool that can base its work on requirements that can help, the way around this is what poetry and pipenv do which is to take something that has these semantic requirements like a requirements.txt and freeze them into a checked-in exact version snapshot of the world (or at least the venv) which is what you later use to reinstall things, and you can periodically update that snapshot to keep track of updates to dependencies and transitive dependencies.

The path forward might be to scrub through the requirements and the pip or conda output and see which dependencies are causing the problem and add or edit restrictions until you can massage the dependency tree back into a working state.

# ? Jul 15, 2021 12:32

CarForumPoster: Jun 26, 2013; â¡POWERâ¡

former glory posted:

I'm pretty new to python and I've hit a major stumbling block in trying to test out an old github repo. It has a requirements.txt with a good 7-8 libraries listed with specific versions, and I've inferred that this was developed on python 3.5. I've tried creating a venv with my system 3.7 python and using pip and it dies trying to install one of the libs.

I tried using a conda environment with all sorts of python versions from 3.4-3.9 -- thinking maybe conda is smarter about this than pip -- and it dies on a different package with each version. There doesn't seem to be a way to get all the requirements to install. I've managed to get slightly different versions of libs to install except for matplotlib and pyclipper now, and I'm wondering if there's some sort of standard way of working this out and maybe I'm just making this hard for no reason? I've tried having PyCharm process the requirements.txt with conda and even manually just installing the packages.

Sorry, don't want to get all stackoverflow on the thread, just wondering if there's some obvious thing I should be using for this type of problem. The whole world of pip, conda, miniconda, anaconda has me reaching all over.

Phobeste explained it well.

Here's how I'd approach the problem. This has worked for me in under an hour but there might be a better way. If you're installing stuff that needs binaries like VC++, CUDA for deep learning, selenium for web scraping, or GraphViz for ML/network visualizations I highly recommend you conda install as much as possible. Google for the package to find it on conda-forge.

Step 1) If the code was written for Python 3.5, which is quite old by now and missing some useful stuff, you should still install that unless you have some other requirements that can't be met by Python 3.5 (e.g. for deploying a web app)

Step 2) I'd look at the requirements.txt, is it in alphabetical order meaning someone made it with pip freeze?

2a) If yes, pip install whatever is imported in the various .py files at the version frozen in requirements.txt. You dont have to have perfect coverage, you just want to install first whatever the authors probably installed first. THEN install requirements.txt, comment out things it dies on by adding a # in front of them in requirements.txt until the install is successful.

2b) If no, install requirements.txt and see where it dies, the comment out things it dies on by adding a # in front of them in requirements.txt until the install is successful.

3) At the very end, manually install the commented out ones, sleuthing on PyPi's history for dates to see what versions of it and its dependencies were published when. If all else fails, install the latest or a later version of the commented out thing with pip or conda. This only works for smallish packages where you can reasonably test for unexpected behavior, as any of those packages and any of their deps may have had API changes.

# ? Jul 15, 2021 14:34

former glory: Jul 11, 2011

Thanks, both posts were really helpful. I'm going to attack it again later tonight with that methodology. I'll keep a snapshot of my entire venv once I finish up my current and first big python project to hopefully avoid this when I inevitably unbox it a while from now.

# ? Jul 15, 2021 20:22

Zoracle Zed: Jul 10, 2001

former glory posted:

Thanks, both posts were really helpful. I'm going to attack it again later tonight with that methodology. I'll keep a snapshot of my entire venv once I finish up my current and first big python project to hopefully avoid this when I inevitably unbox it a while from now.

If you get stuck, post the requirements.txt and I'll take a stab at it (no promises), assuming "a good 7-8 libraries listed with specific versions" means that's the total number of requirements (as opposed to "7-8 of the 75 dependencies have versions pinned").

# ? Jul 16, 2021 01:07

Soylent Pudding: Jun 22, 2007; We've got people!

QuarkJets posted:

It depends on who you're targeting, you can freeze python applications into executables or just have them run on a webserver if you really want to make it easy. For someone cloning your repo, those instructions are sufficient, publishing to PyPi would be better though

Thanks. It's being written as an internal tool but my org decided to open source it once we're done. We're not really interested in hosting it, rather just letting other IT people pull it if they have the same niche use case.

Your comment about publishing to PyPi caused me to learn about setup.py which I think is the answer I didn't know how to ask the right question about.

# ? Jul 16, 2021 02:09

CarForumPoster: Jun 26, 2013; â¡POWERâ¡

Soylent Pudding posted:

Thanks. It's being written as an internal tool but my org decided to open source it once we're done. We're not really interested in hosting it, rather just letting other IT people pull it if they have the same niche use case.

Your comment about publishing to PyPi caused me to learn about setup.py which I think is the answer I didn't know how to ask the right question about.

This post summarizes 90% of PyPi repos last updated before Jan 2021

# ? Jul 16, 2021 03:01

Adbot: ADBOT LOVES YOU

# ? May 13, 2024 21:44

Gobbeldygook: May 13, 2009; Hates Native American people and tries to justify their genocides.

Put this racist on ignore immediately!

I am a coding newbie working through Learn Python The Hard Way. For exercise 36 he says to make a text-based adventure game. I decided to add quicktime events, which requires timed input. I found a complicated, Windows-only method on reddit and it works fine, but I decided to try to make the seemingly simple crossplatform solution work just for it's own sake. Here is what I have:

code:

from threading import Timer

def dead():
    print("You are dead!")

def samurai():
    timeout = 5
    t = Timer(timeout, dead, [])
    t.start()
    print(f"You take one out and engage the other! You have {timeout} seconds per move\n")
    print("HIGH STAB\n1. DUCK\n2. JUMP\n3. PARRY\n4. STRIKE")
    answer = input()
    answer = int(answer)
    if answer == 1:
        t.cancel()
        t.start()
        print("HE\'S OPEN\n1. DUCK\n2. JUMP\n3. PARRY\n4. STRIKE")
        answer = input()
        answer = int(answer)
        if answer == 4:
            t.cancel()
            print("VICTORY!")
        else:
            t.cancel()
            dead()
    else:
        t.cancel()
        dead()
    t.cancel()

samurai()

The way I thought this would work is after the player put in a correct move I could cancel the timer and then start a new timer for the next move. After inputting "1" I get "RuntimeError: threads can only be started once".

Is there some easy way to do what I want or should I stick to my Windows-only method?

# ? Jul 19, 2021 02:27

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›230 »