|
joebuddah posted:I am trying to save data to a custom name to a default location. which os commands were you trying? instead of C:/etc/etc, you should use / and the os.path.join function to build your path: >>> import os # Assuming we want /mydir or C:\mydir, leave the OS decide that >>> target = os.path.join('/', 'mydir') # Check if our target folder exists, if not make it. # Do this for every 'new' folder in your target path, it won't automatically create the entire tree for you. >>> if not os.path.exists(target): ... os.mkdir(target) ... # Move to folder. >>> os.chdir(target) # Print out current working directory. >>> print(os.getcwd()) c:\mydir
|
# ¿ Aug 19, 2016 12:54 |
|
|
# ¿ May 9, 2024 01:51 |
|
joebuddah posted:That worked thank you. one other thing from your snippet, you should use python's context managers('with' statements) for file I/O instead of remembering to try->do thing->finally close everywhere: code:
|
# ¿ Aug 19, 2016 16:27 |
|
baka kaba posted:mkdirs will make all the parent folders though! i should really read docs more often, i never knew this was a thing, ta
|
# ¿ Aug 19, 2016 20:21 |
|
Dominoes posted:Hey, this is probably old news to most of y'all, but using py.test is a clear winner over the builtin unittest. No imports, classes, or inheritance required, just write functions with name starting with 'test', and use an assert statement in the func. Run with 'py.test filename.py' hypothesis has some weird thing about pytest fixtures that i can't remember off the top of my head so i don't use pytest at all, i'd be curious what you're doing that unittest is boilerplate for you compared to writing your own assertion logic for everything though for anyone who works heavily with remote APIs, have you used betamax for unit/integration testing? i only discovered it recently but i'm in love, just wondering if anybody has some horror stories with it or if i'm justified in raving about it to everyone
|
# ¿ Aug 22, 2016 21:13 |
|
Dominoes posted:
Python code:
Python code:
SurgicalOntologist posted:What's it for, I've always wondered. When choosing between two APIs, and one is built around subclassing, I'm choosing the other one 90% of the time. If the advantage is to be able to use things like setup and teardown methods, I'd rather do it pytest's way. If it turns out I need something like that, add a fixture. No need to built up something more complicated from the beginning just because you might need it later. depends mostly on preference, occasionally what you use in testing(that's the thing i couldn't think of earlier), and what your org is like i guess. i work in a mixed environment so xunit-style tests are a plus, if pytest rocks your world then that's cool too edit: also depends on your views on subclassing. a lot of my code is made of subclasses so i don't really have a problem doing it in my tests Dex fucked around with this message at 23:38 on Aug 22, 2016 |
# ¿ Aug 22, 2016 22:35 |
|
SurgicalOntologist posted:But why not just assert a == b? depends on your test data whether that'll take an age. if you can always, always rely on a == b then unittest's assertions have nothing to offer you though, of course. if you do, then you run the risk of screwing up your assertions like i did in the second block and you now have a passing test for something that isn't right, and might not be immediately obvious to anybody why that is. there's other stuff in unittest too, it's just one of the main reasons i like it not really sure what your problem with subclassing is so i'm going to say that's really just a preference thing on your part? i prefer subclassing and using setUpClass and setUp over scoping fixtures but it's not better, it's just how i do things. if unittest offers you nothing and pytest suits your needs, happy days, i just don't think a single import and (unittest.TestCase) is significant enough boilerplate to avoid unittest completely
|
# ¿ Aug 23, 2016 01:22 |
|
SurgicalOntologist posted:I still don't understand. What does assertTupleEqual do differently? From the docs I can't figure out any differences including efficiency. Your zip example makes me think that you have in mind short-circuiting, but a quick test shows that Python does short circuit when testing tuple equality (and I'll bite, the bug was not testing the length of the tuples. Although I still have no idea idea the point you're trying to make as to why you'd ever want to write your test like that). i've seen more or less that assert in a codebase before, is why i included it. did i write it? nope. would i write it? probably, but i'm going to pretend not. would it make through peer review? you'd hope not, but poo poo happens. as it was, that test was green and edited by somebody who didn't _quite_ understand what he was rewriting originally, so the simplified code _should_ have meant a simplified test once it started failing. alas, mistakes were made but green is green. if the standard in place is "use assertTupleEquals when asserting tuples are equal" and so on, it acts as documentation for anybody new scanning through the tests before they do anything. i think this aspect of tests gets overlooked a lot also i just thought asserting 5 == 5 was a bit of a bullshit example when i asked "what does pytest give you that takes a load of boilerplate in unittest", so i figured i was allowed to use one too i've used pytest before and never really had a problem with it, it just doesn't work for me these days so i use unittest. quote:Anyways, I feel like a case could be made that in general, APIs should not be based on inheritance, but I am not prepared to make that case myself. inheritance is about sharing functionality imo, nothing more. if you need functions from three classes in your class, why not just inherit from all of them(assuming your mro isn't haywire)? if those functions could serve the codebase better ripped out of their classes and put somewhere saner, do that instead. if your class requires the user to override stuff and does nothing on its own, then sure, you've probably taken a weird design decision somewhere since that's more of an interface thing. i'm sure somebody else has a fantastic use case for fully abstract classes too, i just don't use any in my own stuff and tend to manage without them quote:I want to find out what these advantages of unittest are that are being implied but not really spelled out. it's built-in, structured nicely and saves me a lot of boilerplate would be the core reasons i like it - i work mostly with json apis now, stuff like self.assertDictContainsSubset is just way simpler to scan through in a PR than converting .keys and .values to sets then using issubset/issuperset, or iterating through expected and comparing to actual, or whatever other method somebody wants to use to say a[keys1-15] are somewhere in dict b and we don't care that the rest doesn't match. if you think the structure sucks and it causes you to write tons of stuff you don't need then that's fine too. i didn't say i thought unittest uses classes for a reason, i said people use unittest for a reason (if it does i'm not aware of it, never really thought too strongly about it tbh), but i do think unittest being class based is an xunit thing more than any core philosophy - which has its benefits when you work with people(or are a person) working on other languages more often than not. i think it was some recent episode of python testing(great podcast even if i'm misremembering why i started thinking about it) that had me considering whether or not the xunit origin is a bad thing for people who work mostly in python though, since a lot of the conventions violate what you'd do in your production code edit: i should refresh pages more often quote:I believe you and Dex when you say it does add something, but I would like to know what that is, for my own education. (having never really used unittest but skipped right to pytest). if you're using pytest and it's all going well you have no real reason to change imo. if you find yourself writing a lot of complicated asserts it might be worth looking at the ones built into unittest instead. that's about it. it can be worthwhile to flip between different test frameworks on different projects just for the sake of it though - doctest is pretty useless for my current work, but it was nice for another project where having the documentation executing itself gave the people using it a degree of confidence that it a) worked b) was documented clearly enough that they didn't have to learn the whole thing just to make a couple of changes later Dex fucked around with this message at 04:19 on Aug 23, 2016 |
# ¿ Aug 23, 2016 04:03 |
|
mike12345 posted:I can't access it from Windows, can access it from inside the vm. I found this blog http://gisellezeno.com/tag/virtualenv.html forwarding from something bound to localhost only like that requires your firewall and selinux policies to allow it. the official docs might make more sense https://ipython.org/ipython-doc/3/notebook/public_server.html#notebook-public-server but yeah, virtualenv has nothing to do with ports which is why google was leading you nowhere
|
# ¿ Aug 27, 2016 17:02 |
|
LochNessMonster posted:I don't think I'll make it multithreaded, definately not straight away. I'm familiar with databases and can make intermediate sql queries so I might take a look at sqlalchemy as well. go for it, if it's big enough that somebody who doesn't care is going to need to scroll past it then maybe make a project.log thread and just link it here when you want feedback. the sqlalchemy tutorial uses an in-memory sqlite database, so it's worth checking out http://docs.sqlalchemy.org/en/latest/orm/tutorial.html to get familiar with how ORM works. you can of course just use raw sql queries everywhere if you want to, but that's no fun
|
# ¿ Sep 3, 2016 12:16 |
|
LochNessMonster posted:Thanks for the advice. I did look at the project.log but felt kinda silly for starting one as it's my first ever project. first project or five thousandth, there's no entry criteria for project.log. start with functions, classes can come later(if they ever need to). try to break down your functions until they're doing one thing you can easily test within them, and/or replace easily later. for example, if my_parser.py contains five functions that have absolutely nothing in common other than being generally related to parsing, then there's no need for a MyParser class. once you start breaking things up in a way that makes sense to you, you'll figure out for yourself how your code should be structured. there's no hard and fast rules around "this MUST be a class", and you could easily write an entire app without using any. also do yourself a favour and try to get in the mindset of test driven development. it might seem like serious overkill for a first project("why would i need a test for something that just returns a url string? waste of time!"), but i find writing tests up front helps me reason about the flow of an application a lot easier than banging out a bunch of code that just does it first and trying to make it sane later.
|
# ¿ Sep 3, 2016 13:04 |
|
LochNessMonster posted:Any guides/tutorials/guidelines I can look into on that? What I've done so far is just test every minor thing I'm trying to build as a seperate standalone app with static and/or test data. the official docs are a good starting point: http://docs.python-guide.org/en/latest/writing/tests/ . basically let the code tell you my_cool_function is returning 'i am very cool' by calling it and asserting the value, instead of printing it out and reading it yourself. this is one of the reasons why TDD helps me design my applications, too - instead of "oh i need an app that does x, y, z and also saves the world", break it down to one function you need _now_, write tests for that, then implement it. don't get too hung up on which testing framework you want to use. i use unittest most of the time, because it suits me. if you experiment with a few of them and find one that fits you better, roll with it quote:I'm not sure how vcr will help me do this easier (without making it too complex for me). Looking at your code snippets confuses me as I have no idea how to use it in my current code. I really am a beginner at the bottom level here. i actually posted in this thread earlier asking about https://betamax.readthedocs.io/en/latest/ , which is based on vcr. it's very, very cool if your app makes a lot of web requests and you don't want to use stubs or mocks, but might be a bit in-depth if you're not up on unit testing your code yet. keep in mind for now i guess, but the basic idea for your use case would be: - you have a function that gets a web page, parses it, then fetches the other eight pages(making up numbers, shh) - if you have a unit test for this function, it's going to run really slowly since you're waiting for nine network requests to complete - if you're using vcr/betamax and you've already run that test, the requests(headers, bodies, status codes, the lot) are already written to disk so it'll just replay those instead of making actual web calls so, an automatic way of doing what you're already doing by saving the files locally. from onionradish's example, synopsis.yaml would contain the request for 'http://www.iana.org/domains/reserved'.
|
# ¿ Sep 3, 2016 16:08 |
|
i've never used pyaudio, but from a quick look, instead of blocking during recording you can use a callback: http://people.csail.mit.edu/hubert/pyaudio/#wire-callback-example
|
# ¿ Sep 12, 2016 11:00 |
|
JVNO posted:I'm curious why the blocked form of the code wouldn't be valid? The intention is to record the entire PyAudio stream from the experiment (up to about 30 minutes) and dump it to a single wave file. i'm just guessing from the code you posted(it's monday and i'm ill so reading is for nerds), but in her example she's reading before the questions are presented. in the code block you posted, you're reading afterwards. the 'hang' you're referring to is possibly just the recording actually starting after the questions have been posted, since: Python code:
|
# ¿ Sep 12, 2016 22:04 |
|
LochNessMonster posted:Thank you, this what I had in mind as a concept, but I had no clue if it would be a good and/or efficient way of doing it. I also didn't know what type I should've used. Thanks for helping me figuring this out! you could also use sqlalchemy for this. first create your database entity: Python code:
Python code:
|
# ¿ Sep 16, 2016 11:32 |
|
LochNessMonster posted:I'm pretty sure I'll be running into a lot of design flaws in a while, but figuring those out seems like half the fun. i don't think i've ever got through an entire project without rethinking something i did at the start and could have done better. getting something working and being happy with how it works are two totally different things, i barely ever accomplish the latter. it's good to know when to just leave something be though otherwise you'll never finish anything! sqlalchemy is definitely overkill for what you're trying to do, it's just handy to keep in mind if things start growing. the more you understand using database connections and sql directly, the easier the sqlalchemy stuff is to figure out, though, so there's no real rush to start using it - was just throwing it out there as an option to look into if you're curious
|
# ¿ Sep 16, 2016 20:10 |
|
LochNessMonster posted:yeah, you're right. It's a list containing dictionaries. since you're iterating through a list, just use arcticzombie's example for each dict in it: Python code:
|
# ¿ Sep 21, 2016 12:14 |
|
py3's open function handles unicode encoding for you, py2 does not. the codecs package can do it though:Python code:
$ cat output.txt £
|
# ¿ Sep 27, 2016 18:51 |
|
LochNessMonster posted:I was considering this when looking into the differences between django and flask. And making each functionality a microservice (sorry for buzzwords) actually sounds like a great idea since I have no clue what I'll be doing next. It's just a hobby project I started and keep expanding. Making components small and (relatively) easy to change sounds like a good idea. That said, I'll first try and get a default flask application running. After that I'm gonna experiment with a RESTful api. When I'm ready I think I'll need to head over to the front end thread and start a religious war by asking whether to use React or Angular. falcon is pretty handy for rest apis
|
# ¿ Oct 18, 2016 08:18 |
|
huhu posted:After hours of frustration with a script I'm writing, I've come to realize I could have been using CSVs instead of Excel files. So, lesson learned, if you can deal with CSV files instead of Excel do it. If you're wondering why, Excel columns can start at 1 or "A" however everything in Python starts at 0. i use openpyxl for excel stuff
|
# ¿ Nov 28, 2016 19:13 |
|
[flake8] max_complexity = 10 max_line_length = 9999
|
# ¿ Dec 1, 2016 14:28 |
|
lifg posted:Does anyone have experience or recommendations on building Python software with virtualenv and Docker? what are you using for your builds? we use a swarm and yet another docker plugin to run our builds - each build spawns a brand new container so there's no need for virtualenv. if you're intending to use docker compose to stand up a bunch of build slaves that'll be persisted for more than a single build, you'd want virtualenv
|
# ¿ Dec 3, 2016 19:11 |
|
i haven't used openshift but i see you have a couple of options there - custom dockerfiles, hooking in with jenkins pipelines(which i've worked with _a lot_), and more. so it's really a question of how you want to work. if you just want to code in your ide and run your tests etc locally, and ignore the final artifact that pops out of your build system, then you don't need docker locally. your build process could just be running a custom dockerfile that installs your deps and runs your unit tests as part of every commit, but only pushes the image on merges to master. so something likecode:
Dex fucked around with this message at 22:39 on Dec 3, 2016 |
# ¿ Dec 3, 2016 22:13 |
|
from some stuff i looked at last year, go's concurrency for cpu intensive threads was around 50x cpython's. however, go has some seriously nasty leaks that are completely beyond your ability to control, so you will almost certainly end up with memory issues that are near impossible to isolate, so if you have even the slightest reason to care about memory consumption, it seems like a terrible choice i'd aim for pypy before running python in go if i was willing to make tradeoffs on certain libs since it's apparently significantly more performant than cpython, but i don't actually need to so i've only done some light reading on it
|
# ¿ Jan 4, 2017 20:53 |
|
Popen works as a context manager, fyi, you can use "with Popen(blah) as proc:" to get your waiting and file descriptor closing and so on for free
|
# ¿ Jan 13, 2017 10:36 |
|
socialsecurity posted:I've started looking into this but it seems it would require quite a bit of rewrite I didn't think sending a json back to a html post would be this much of an undertaking but thank you I think in the end going down this path would be easier to work on down the road. falcon is my go-to for quick and easy apis: Python code:
code:
Dex fucked around with this message at 02:38 on Jan 29, 2017 |
# ¿ Jan 29, 2017 02:35 |
|
|
# ¿ May 9, 2024 01:51 |
|
Boris Galerkin posted:I've been trying out pytest and it's really neat. My test codes look much cleaner than with unittest, but what I like the most is the colored red/green output. I do something like TDD so I'm constantly alt-tabbing or tmux pane switching etc to another terminal to re-run tests. Is there already an established way to tell pytest something like "monitor test folder x and re-run your tests every single time source code in folder y changes"? pytest-xdist gives you a watch option(-f). quote:And going further (but maybe too Vim specific) is there a way to have pytest color my vim status bar red/green depending on status so I don't even have to look over to the other terminal unless I need to read the status messages? i haven't used vim for actual dev in a while, but are you using some plugin like python-mode or just rolling your own config? maybe do something like run pytest on save and check for v:shell_error instead of watching? personally i just use pycharm and toggle auto-test on though
|
# ¿ Mar 5, 2017 16:42 |