|
Since there's not a thread for general/language agnostic programming questions and I'm most comfy with Python, I figure I'll put this here: I'm having to do some writing for work - writing curriculum to introduce newbies to Python and JS. Right now I'm writing a section on writing/testing algorithms, and specifically about identifying edge cases and writing code to handle them. "Edge case" in this example covers edge cases, corner cases, boundary cases, etc. - basically any input to a function that needs to be handled in some special way. Is there a term for all cases that are not anomalous? I'm pretty sure the term is "common case" - I've read it in a few books, and it seems to fit, but I'm very explicitly trying to not come up with my own special terminology that would confuse students when they look at other resources later, so if "common case" actually has another common definition I probably shouldn't use it.
|
#
?
Jan 9, 2022 23:58
|
|
|
|
| # ? May 14, 2024 07:44 |
|
|
QuarkJets posted:Hell yeah On this note, every time I try to understand "modern" Python packaging and build systems, I end up more confused than ever. At work, we exclusively use SCons and a custom/obscure package management system on top of conda (many of the constituent packages use pybind11). SCons seems to be out of favour these days (I don't know if it's for any particular reason, but I never put in the time to understand it), so I tried Meson for a change, and it does seem pretty nice. It was much less painful to work with than pybind11's CMake examples. Still, it's not quite ubiquitous yet, and it seems like overkill for a small pure Python package. For that purpose, I also tried poetry with a pyproject.toml, and while it also seems to work, I'm not sure I understand how the dependency resolution works, how I as a developer am supposed to determine what the oldest possible version of a dependency with a fluctuating API (numpy, matplotlib, scipy etc) is, how users should install v0.1 of the package before I send it off to PyPI, etc etc.
|
|
#
?
Jan 10, 2022 00:39
|
|
|
Poetry is really just a higher-level interface for setuptools; both support a pyproject.toml. You can specify your dependencies explicitly and add version ranges that you think are reasonable. There's no crystal ball here of course, so it's up to you whether you want to limit by major version change or just leave things loose. If your package becomes super popular then that's when you use CD to start distributing conda packages and really flushing out dependencies. Until then, there's no point in trying to account for people who absolutely refuse to stop using Python 3.2 or whatever, you just need to specify ranges that you feel are reasonable as a starting point and go from there. I don't understand the last question, ideally you publish package PyPi and that's how users install it - with `pip install`. If someone needs a pre-release build just send them a wheel (iirc this is `poetry build`, as opposed to `poetry publish`).
|
|
#
?
Jan 10, 2022 00:57
|
|
|
qsvui posted:Sounds like Working Effectively with Legacy Code by Michael Feathers might help. It doesn't use Python but I think the advice should be generally applicable. Book's on my buy list, but I suspect most of it won't be applicable; this codebase is only a year or two old, it's just big enough to be a pain to retrofit testing in. Edit: Looking around, actually I think this book is actually probably the right answer. Thanks! I picked it up. Falcon2001 fucked around with this message at 02:36 on Jan 10, 2022 |
|
#
?
Jan 10, 2022 02:32
|
|
|
QuarkJets posted:I don't understand the last question, ideally you publish package PyPi and that's how users install it - with `pip install`. If someone needs a pre-release build just send them a wheel (iirc this is `poetry build`, as opposed to `poetry publish`). Oh I wasn't clear, I mean for a user to install a package that has yet to be published on PyPI but has source available on github, normally you'd clone and `python setup.py install` (`--user` perhaps). If you provide a requirements.txt generated by poetry (or otherwise) I think you can do `python -m pip install .` instead?
|
|
#
?
Jan 10, 2022 05:46
|
|
|
eXXon posted:Oh I wasn't clear, I mean for a user to install a package that has yet to be published on PyPI but has source available on github, normally you'd clone and `python setup.py install` (`--user` perhaps). If you provide a requirements.txt generated by poetry (or otherwise) I think you can do `python -m pip install .` instead? Yup, you don't even need a requirements.txt; if you have a pyproject.toml you can just pip install the local directory.
|
|
#
?
Jan 10, 2022 10:44
|
|
|
Can setup.pyless packages be installed editable (pip install -e .) yet? This is the only reason my packages still have a minimal setup.py with the actual stuff in setup.cfg/pyproject.toml.
|
|
#
?
Jan 10, 2022 22:57
|
|
|
ArcticZombie posted:Can setup.pyless packages be installed editable (pip install -e .) yet? This is the only reason my packages still have a minimal setup.py with the actual stuff in setup.cfg/pyproject.toml. PEP 660 is accepted but hasn't been implemented yet, unfortunately.
|
|
#
?
Jan 10, 2022 23:22
|
|
|
I had no idea Netflix used Python for so much of their backend. Back on the weird work project lane, I'm working on using Flask to re-develop my current suite of Excel calculators into a Flask app. I'm finding it, thus far, much easier to parse and work with than Django. Next step would be putting it on the company intranet and expanding the functionality to things other branches do that we don't. Make myself redundant as the breakdown guy but become full-time web services dev 
|
|
#
?
Jan 11, 2022 21:14
|
|
|
I want to import from within a package but keep full namespace to avoid conflicts with STL. So far I do not see any way to do this. For example, if I have project/package/mail.py and project/package/util.py. If my main app is project/app.py I can do "import package.mail" and that namespace is fine, no conflicts. But if I need to import package.mail.py in package.util.py (for example for type hinting an argument to a function) I seem to be stuck with relative imports, e.g. "from . import mail" and at that point it's in conflict. Is there a way to do this? I'm wondering if there isn't and maybe it's because of risk of circular imports. I normally use empty __init__.py files and don't do anything fancy with imports, just try to keep it simple and don't mind the slightly longer length that results.
|
|
#
?
Jan 12, 2022 01:22
|
|
|
mr_package posted:I want to import from within a package but keep full namespace to avoid conflicts with STL. So far I do not see any way to do this. For example, if I have project/package/mail.py and project/package/util.py. If my main app is project/app.py I can do "import package.mail" and that namespace is fine, no conflicts. But if I need to import package.mail.py in package.util.py (for example for type hinting an argument to a function) I seem to be stuck with relative imports, e.g. "from . import mail" and at that point it's in conflict. No good solutions, some hacks. You should be able to do a relative import with rename, from . import mail as _mail. You can make this an absolute import by appending ~/package/ to sys.path dynamically at the start of util file if you want.
|
|
#
?
Jan 12, 2022 01:55
|
|
|
ArcticZombie posted:Can setup.pyless packages be installed editable (pip install -e .) yet? This is the only reason my packages still have a minimal setup.py with the actual stuff in setup.cfg/pyproject.toml. Yes. None of my editable packages have a setup.py. I don't know how long ago that became possible, but I've been doing it since at least when Python 3.7 was the latest version OnceIWasAnOstrich posted:PEP 660 is accepted but hasn't been implemented yet, unfortunately. Maybe you can't just have a pyproject.toml on its own, but a setup.cfg + pyproject.toml works
|
|
#
?
Jan 12, 2022 06:12
|
|
|
mr_package posted:I want to import from within a package but keep full namespace to avoid conflicts with STL. So far I do not see any way to do this. For example, if I have project/package/mail.py and project/package/util.py. If my main app is project/app.py I can do "import package.mail" and that namespace is fine, no conflicts. But if I need to import package.mail.py in package.util.py (for example for type hinting an argument to a function) I seem to be stuck with relative imports, e.g. "from . import mail" and at that point it's in conflict. If `package` is installed then you should be able to import its full namespace from anywhere, without the need for relative imports. Try installing it in the editable mode that was being discussed in the posts before yours
|
|
#
?
Jan 12, 2022 06:17
|
|
|
Ok I'm messing about with a monorepo for microservices and really trying to get the hang of it both with PyCharm and other IDEs. Here's a contrived example: pre:. ├── applications │ └── app1 │ └── independpentjob1 │ ├── requirements.txt │ ├── independentjob1.Dockerfile │ ├── independentjob1.py │ └── tests │ └── test_independentjob1.py └── services
|
|
#
?
Jan 12, 2022 19:28
|
|
|
I’m not sure I understand your idea fully, but PyCharm supports arbitrary number of interpreters per project, and you can have file-level run configurations (arbitrary number of those per file). In “global” mode, e.g., for the purposes of terminal or naive executions of code, it supports only 1 interpreter being considered active at a time.
|
|
#
?
Jan 12, 2022 19:34
|
|
|
cinci zoo sniper posted:I’m not sure I understand your idea fully, but PyCharm supports arbitrary number of interpreters per project, and you can have file-level run configurations (arbitrary number of those per file). In “global” mode, e.g., for the purposes of terminal or naive executions of code, it supports only 1 interpreter being considered active at a time. I'm not sure I understand my idea fully either. Someone on my team suggested using k8s for a new project and I'm trying to figure out how this could work. At the end of the day, KISS with a monolith has a ton of runway, or I could go whole hog and just do everything with FaaS and get nearly the same thing. Still, trying this on. I was able to figure out how to add additional interpreters that weren't project default (it's been a while) and that part is working just fine. I'll figure out if I can be smarter than "global" mode next. Thanks!
|
|
#
?
Jan 12, 2022 20:22
|
|
|
Is there a way to enforce the number of arguments an abstract method will accept? Like Python code:I want to enforce that all run() methods in any subclass objects accept a single input argument (simdata) to be compatible with the rest of the code.
|
|
#
?
Jan 13, 2022 04:11
|
|
|
Nohearum posted:Is there a way to enforce the number of arguments an abstract method will accept? There are some (imo cursed) ways to do that by abusing the fact that abstract methods in Python can have an implementation, and so you just do some argument checks via super(), but I would challenge you to a sunset duel on swords if this was something I’d discover in a production code I have to touch. The language design simply does not seek to support for this style of programming - abstract methods are tracked by name only. What I would suggest doing instead is that you: 1) rename run to _run 2) implement _validate_run 3) implement run as executing 2, then 1 Hed posted:I'm not sure I understand my idea fully either. Someone on my team suggested using k8s for a new project and I'm trying to figure out how this could work. At the end of the day, KISS with a monolith has a ton of runway, or I could go whole hog and just do everything with FaaS and get nearly the same thing. Still, trying this on. I’m sorry for not being too helpful here as this goes way past my preferred amount of community tech support, but k8s is just a deployment option, one paired well with containerised development environment. If this is not a horizontally scaled service, and a micro service mesh instead (which it sounds it is, as you care about having a zoo of interpreters), you should just do one repo per service and avoid inflicting upon yourself Google problems for no clear reason.
|
|
#
?
Jan 13, 2022 04:57
|
|
|
You might try Vigil, it has... superior support for declaring absolute requirements Better answer: state your desires in the docstring and move on
|
|
#
?
Jan 13, 2022 06:50
|
|
|
It's too bad you can't just do:Python code:
|
|
#
?
Jan 13, 2022 07:08
|
|
|
In thinking about my issue previously I started to look at importing from a parallel file within a package as a bit of a code smell: it should either be in a sub-package or it should be in the same module. Ok. So I can make sub-packages and import them to the main package and then import that into my app for use. In my test project I have three files: /project/app.py /project/package/wrapper.py /project/package/apis/mail.py I want to use "import package.wrapper" in app.py. I want to use "import apis.mail" in wrapper.py. I do not mind long namespaces and got into the habit of importing this way a few years ago mostly due to google style guide. That means I import pathlib and call pathlib.Path() all over the place instead of doing "from pathlib import Path" but I'm used to it. It seems like it should be simple enough but this is quite broken: if I add "import apis.mail" into /project/package/wrapper.py it works if I run wrapper.py directly. But when I go into my main app, /project/app.py, and import package.wrapper, it now dies with ModuleNotFoundError. I thought it seemed reasonable to make these namespace packages but this also fails. I would like to pull this off with empty __init__.py files or none at all (namespace packages) so I don't have to maintain them if I make a change but will do so if that is the answer and/or just the best way to do things. This doesn't seem like it should be quite this hard; I see examples of packages with sub-packages all over the place. I can't make it work. I'm wondering if Zen of Python re: namespaces is some kind of trick and doesn't apply to the import system and I should just be pulling in classes directly. Or, I put the main wrapper class in app.py. But this kind of defeats the purpose; I want to have the package separate from the app logic so I can re-use for example on other OS which may have slightly different behavior (I deploy win/mac/linux). If the bottom line is this is the wrong way to do it with Python I'm happy to change to the right way I just cannot figure out wtf that is. Hopefully something simple I'm just not grokking. edit: found a SO that describes pretty much same issue/idea: https://stackoverflow.com/questions/67609395/python-subpackages-and-namespaces I'll see if I can get it working well by using imports in __init__.py. I've done some testing and it hasn't worked for me but I assume if I import the classes directly it may work.. edit2: I _think_ part of the problem is the __init__.py files are executed by the calling process and that is why I get different results calling /project/package/wrapper directly vs /project/app.py. I often do this for testing but it may be I can get it working if I only ever run the import from the main app and then runn app.py (or any other module that imports /project/package/wrapper). I'm really surprised there doesn't seem a way to do imports just using a logical directory structure but I'll assume there's good reasons for it under the hood. mr_package fucked around with this message at 22:59 on Jan 13, 2022 |
|
#
?
Jan 13, 2022 21:48
|
|
|
mr_package posted:In thinking about my issue previously I started to look at importing from a parallel file within a package as a bit of a code smell: it should either be in a sub-package or it should be in the same module. Ok. So I can make sub-packages and import them to the main package and then import that into my app for use. Your problem is that you're not adding your package to PYTHONPATH correctly. Your Python executable has no magical knowledge of packages that haven't been formally installed, for absolute imports you’re trying to use. Apologies for lazy screenshot reply: 
cinci zoo sniper fucked around with this message at 23:03 on Jan 13, 2022 |
|
#
?
Jan 13, 2022 22:55
|
|
|
I don't quite understand why I need to "install" a package-- doesn't Python traverse the dir looking for them? That's how namespace packages work, yeah? (Screenshot is great, thanks-- I find ascii dir trees hard to read actually).
|
|
#
?
Jan 13, 2022 23:03
|
|
|
mr_package posted:I don't quite understand why I need to "install" a package-- doesn't Python traverse the dir looking for them? That's how namespace packages work, yeah? For relative imports (starts with a full stop) it will traverse the local directory, for absolute imports (your choice) it will traverse sys.path. https://docs.python.org/3/library/sys.html#sys.path You can simply import and print it out to see what’s going on with yours in your dev environment.
|
|
#
?
Jan 13, 2022 23:11
|
|
|
mr_package posted:In thinking about my issue previously I started to look at importing from a parallel file within a package as a bit of a code smell: it should either be in a sub-package or it should be in the same module. That is not a code smell. Why would it be? Long namespaces are actually a smell, one you can solve using __init__.py though mr_package posted:Ok. So I can make sub-packages and import them to the main package and then import that into my app for use. If "package" isn't in one of the directories in the "sys.path" list, then you won't be able to import it. You can either be lazy and gently caress with sys.path or PYTHONPATH yourself, or you can do the proper and recommended thing and pip install your package (and there is even the -e option to make your installation link against your codebase, editable installs exist so that you don't have to modify your path by hand)
|
|
#
?
Jan 13, 2022 23:46
|
|
|
QuarkJets posted:That is not a code smell. Why would it be? Anyway for the rest of this I had always just imported from the live filesystem, from the project dir. I have never installed a package I wrote and it's always worked. But I've hit this wall with the import syntax and it seems it's because I'm trying to write/use/import packages without installing them. It works well enough for simple stuff but I guess the luck runs out when you start to doing something a little more complex.
|
|
#
?
Jan 14, 2022 00:13
|
|
|
For what it’s worth, I’ve not installed a package I’m developing even once in my career. Dynamically injecting path variables in my opinion is a hack, but I prefer to keep it that for unfinished packages, when necessary.
|
|
#
?
Jan 14, 2022 00:17
|
|
|
The editable package install is super useful for something like pulumi, where in order to use the tool you'll have dozens of isolated python programs that get invoked by a CLI utility, but you'll also want some way of defining and importing abstractions into those programs. If you use an editable package you don't have to juggle relative paths to get to your shared code in each program, and changes to the programs and the shared code can also be tightly coupled in a monorepo, which saves you from tedious and error prone version bumps and staged releases.mr_package posted:If you have two parallel files [...] I think you're correct about this, if you can consider something as being "parallel" to something else there are probably better facilities for handling it than putting it into its own module. I don't spend a lot of time working on super large codebases but IMHO namespaces are useful as far as that they provide a descriptive name, so you should use them exactly until the descriptive name stops being useful. I don't think there's anything inherently valuable about, for example, Online.FoodOrders.Delivery vs Online.FoodOrders.Pickup. You could just have them be OnlinePickupOrder and OnlineDeliveryOrder.
|
|
#
?
Jan 14, 2022 00:27
|
|
|
Don’t get me wrong, I’m not calling the editable installs bad or inefficient. I just personally dislike the approach on a philosophical level, and have yet to encounter problems where I’d deem it mandatory - which was to nudge mr_package to reconsider his options. Or to just sidestep the roadblock entirely - there’re plenty of very high quality Python packages that have been developed using relative imports exclusively. Again, maybe not my own preference, but sklearn is a complex and well written project where that works just fine.
|
|
#
?
Jan 14, 2022 01:09
|
|
|
mr_package posted:If you have two parallel files and one is importing the other it's either related enough it should just be part of same module, or separate enough it should be a sub-package. I'm not claiming this as gospel, just something I started noticing / thinking about while working on this. Why, though? What makes this a code smell, in your opinion? I like to break up modules if they start to get too long, and 1 new module isn't enough to justify a sub-package imo
|
|
#
?
Jan 14, 2022 05:22
|
|
|
QuarkJets posted:Why, though? What makes this a code smell, in your opinion?
|
|
#
?
Jan 17, 2022 19:58
|
|
|
That kind of sounds like a prescription for turning all projects into one big file, though? I don't think that's what you want, but it's the conclusion that my head makes so I'm trying to get more clarity Like if I had a main.py file that was over 1k lines split among various functions, I might think to myself "let me organize this a little bit by putting all of the io functions in a new file named io.py." Would you say that is a code smell, or no?
|
|
#
?
Jan 17, 2022 21:19
|
|
|
Yeah, I'm with QuarkJets here on this one. Namespaces were invented for a reason.
|
|
#
?
Jan 17, 2022 21:28
|
|
|
Funny, I'm the opposite of cinci zoo sniper, the first thing I do in every project is create an empty environment and pip install -e the directory. And yes, I always use absolute imports.
|
|
#
?
Jan 17, 2022 22:36
|
|
|
Can anyone help with this hopefully simple question? I'm adding logging to my simple script with these two commands cobbled together from googling: Python code:
|
|
#
?
Jan 18, 2022 16:16
|
|
|
fuf posted:Can anyone help with this hopefully simple question? You can add a formatter to the StreamHandler: Python code:
|
|
#
?
Jan 18, 2022 19:27
|
|
|
dorkanoid posted:You can add a formatter to the StreamHandler: Perfect, thanks so much 
|
|
#
?
Jan 19, 2022 11:31
|
|
|
Ideally if you want multiple types of logs you don't use basicConfig, you use StreamHandler and FileHandler (or whatever else, like RotatingFileHandler) and give a common formatter to each The logging module also supports config files, which is great
|
|
#
?
Jan 19, 2022 20:24
|
|
|
It's ridiculous how  I get over a simple half-functional login page in Flask that centers the form on the screen and pins the header to the bottom. If I can get this poo poo on a Heroku instance and spread it around the company I'm gonna be quite happy and make a new name for myself other than "the Excel toucher" and "the NAV geek".
|
|
#
?
Jan 19, 2022 20:48
|
|
|
|
| # ? May 14, 2024 07:44 |
|
|
Functional apps are cool, assuming you can ignore the duct tape and broken dreams that holds most of them together of course.
|
|
#
?
Jan 19, 2022 21:00
|
|
