Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
fuf
Sep 12, 2004

haha
Is Tango with Django still recommended? Anything better? Has to be text because I'm learning on the sly at work.

Any random tips for setting up a development environment for learning Django? It's always the worst bit for me. I want to do the fun programming part but first I have to install a load of confusing stuff. :(

Since I'm learning at work and at home I think it makes sense to do it all online somewhere - the OP recommends heroku. I was thinking that DigitalOcean probably also has a django "droplet" ready to go...

Adbot
ADBOT LOVES YOU

susan b buffering
Nov 14, 2016

The one click droplet on DO is hilariously out of date last I checked, but a $5 droplet can run django just fine.

I’d recommend gunicorn for your WSGI server along with nginx. The gunicorn docs have a guide for this. Also definitely use a virtual env.

Deployment can be somewhat tricky but it’s worth figuring out. You can also just run it locally until you feel more comfortable.

fuf
Sep 12, 2004

haha
Thanks. I don't really want to run anything locally on my work PC if I can help it...

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

fuf posted:

Is Tango with Django still recommended? Anything better? Has to be text because I'm learning on the sly at work.

Any random tips for setting up a development environment for learning Django? It's always the worst bit for me. I want to do the fun programming part but first I have to install a load of confusing stuff. :(

Since I'm learning at work and at home I think it makes sense to do it all online somewhere - the OP recommends heroku. I was thinking that DigitalOcean probably also has a django "droplet" ready to go...

Django has some of the best open source docs out there. I'd start with them and the official tutorial.

necrotic
Aug 2, 2005
I owe my brother big time for this!
If anyone starts seeing issues with pip install binary packages the 20 release has a bug that breaks the "manylinux" platform. I had to downgrade to 19 to get our CI back up.

CarForumPoster
Jun 26, 2013

⚡POWER⚡
Youtube recommended me some python-based "Technical interview with a {tech_company} engineer" videos and I watched a couple.

When you all interview do you really ask people about time and space complexity of python functions and like...implement the reverse of some string?
Do you expect them to use few or no packages?

For actual coding skills I wanna know that they can manipulate json, talk to APIs and properly handle data (e.g. use pandas). For less technical but important skills I want to make sure they've got some experience working on team projects where other people need to read their code months later.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

For me, the most important part of interviewing an applicant is being able to see code they've already wrote. It's really helpful to be able to see some projects they've put up on github or wherever.

Otherwise it mostly seems like I'm just screening for people who've read Cracking the Coding Interview.

I'm mostly not even really concerned with a lot of specific technical skills. If you're a good programmer but you've never touched JSON or worked with a web API, getting up to speed with any of that is pretty trivial. Of course, if it's a completely different area it's a somewhat different story...like if you've never done any sorts of ML or stats stuff it's going to take a lot longer to get up to speed with that. But if they've done ML in some other framework or language that's not a big issue.

KICK BAMA KICK
Mar 2, 2009

fuf posted:

Is Tango with Django still recommended? Anything better? Has to be text because I'm learning on the sly at work.

Any random tips for setting up a development environment for learning Django? It's always the worst bit for me. I want to do the fun programming part but first I have to install a load of confusing stuff. :(

Since I'm learning at work and at home I think it makes sense to do it all online somewhere - the OP recommends heroku. I was thinking that DigitalOcean probably also has a django "droplet" ready to go...
Will take any opportunity to re-plug Test-Driven Web Development with Python aka Obey the Testing Goat; it's intended as an introduction to TDD, obviously, but it doubles as a really solid intro to Django. Goes all the way from setting up an environment, like you asked about, through live deployment. Don't worry too much about the Django version being old; I read it while working on a Django 2 project and I don't think there was a single major thing that wasn't almost exactly the same.

amethystdragon
Sep 14, 2019

Grump posted:

Can anyone recommend me tutorials that walk you through implementing an authentication server from start to finish, preferably in Python?
I would also recommend Django with the oauth2 module.

Thermopyle posted:

A few years ago I did this exact thing and man it's irritating. I was never able to find a good single source of information. I ended up having a dozen tabs that I'd reference while trying to figure out why all of them contradicted each of them in some small way.
As suggested above, I cannot seem to find a single source.

I would recommend taking a look at the standard Django tutorial for building apps and then seperatly the documentation for the oauth2 module. When I did it about 5 years ago there was some tweaking I needed to do to fully get it working (and so people could auth with more then one oauth source). But sadly the code was for a job and thus private so i cannot post it...

Dominoes
Sep 20, 2007

In the spirit of MLK day, disobey. The goat's a tyrant who aims to steal your time make your code resistant to change. Test when appropriate; don't let it drive your project.

Along the lines of Therm and amethyst's posts, start with the official tutorial. If you have trouble or questions, post here. Django has a large surface area, and you may not need big parts of it. I think it's worth learning Django RestFramework concurrently.

Dominoes fucked around with this message at 17:38 on Jan 22, 2020

breaks
May 12, 2001

Anyone know of any reasonably well tested projects on GitHub using starlette/Ariadne/mongo or at least a similar async graphql stack? Just want to see how other people are going about testing it before committing to anything in particular in that regard. Any other potentially relevant resources, articles, whatever would be welcomed, too.

NinpoEspiritoSanto
Oct 22, 2013




Dominoes posted:

In the spirit of MLK day, disobey. The goat's a tyrant who aims to steal your time make your code resistant to change. Test when appropriate; don't let it drive your project.

Along the lines of Therm and amethyst's posts, start with the official tutorial. If you have trouble or questions, post here. Django has a large surface area, and you may not need big parts of it. I think it's worth learning Django RestFramework concurrently.

If you or anyone have any recommendations for TDD rebuttals I'd be interested. It's all the rage at work and plenty of "it's the only way" claims.

necrotic
Aug 2, 2005
I owe my brother big time for this!
If you're writing a bunch of mocks you're doing it wrong. The TDD I've seen involves a lot of mocking that ties the tests way to closely to any implementation of the overall codebase.

mr_package
Jun 13, 2000
Someone recommended 'Domain Driven Design' as an alternative approach but I haven't bought/read it yet.

Dominoes
Sep 20, 2007

Bundy posted:

If you or anyone have any recommendations for TDD rebuttals I'd be interested. It's all the rage at work and plenty of "it's the only way" claims.
This Jonathan Blow video hits the main points. One of the bigger points for me is that refactoring (eg to make the code easier to work with in the future, correct poor decisions etc) has the potential to break many tests, which makes it tempting to not do it. I find tests to be situational. Ie, I have a project that depends heavily on parsing and creating strings in specific formats; testing has made it easier to manage, and for a number of tasks, I took the TDD-approach of writing the test first/concurrently with the code. If you have shaky/complex logic, that's another example when you'd want to test thoroughly. GUI code? I'm not sure what unit testing would mean.

I find testing hampers creativity, especially in the early phases. Ie I enjoy trying things, deleting chunks of code, rewriting etc; TDD would make this more cumbersome. I'm suspicious that TDD is something that originated in enterprise code, where it helps deal with multiple coders with different styles and skill levels that may not directly interact. Or deliberately-designed code designed to meet specific requirements. I'm also suspicious it was related to dynamic type systems (eg earlier versions of Python) as a way to catch bugs before runtime hits a given branch.

Dominoes fucked around with this message at 00:37 on Jan 24, 2020

KICK BAMA KICK
Mar 2, 2009

I'm just a novice/hobbyist and am not rigorously following write-the-tests-first methodology but I did find TDD concepts useful in a few ways -- using tests as a roadmap toward implementation, making sure I handle all possible errors or edge cases, simply reminding what I left undone if I get interrupted, and as a motivation to write testable functions/methods that correctly separate concerns and Do Exactly One Thing. There's a lot of mocks in the tests for the project I'm working on cause my code scrapes some stuff/uses some external APIs but there's a point where if I find myself mocking too many things in a test, I take note that the function being tested is probably doing too much at once.

However, I did just the other day hit my first case of a test tricking me into thinking something was OK that wasn't -- I changed the signature of a function, adding a required argument, and failed to add that argument in the one place it got called cause the tests covering the caller mocked the function I changed and made assertions about its call arguments against the old signature; I didn't notice/forgot to change those tests when I changed the signature. (Not 100% on why PyCharm inspections didn't catch that but honestly I might have only run them against uncommitted changes and not touched the module doing the incorrect call.) Not really the tests' fault -- not like I would have caught it any sooner without tests -- and they've absolutely prevented way more errors than they've masked so I'm sticking with it for now. Can imagine they get less useful as I gain experience and internalize some of these lessons without needing the tests to guide me there but at the moment I don't think I've ever learned as much from any coding text as I did from The Goat.

necrotic
Aug 2, 2005
I owe my brother big time for this!

KICK BAMA KICK posted:

There's a lot of mocks in the tests for the project I'm working on cause my code scrapes some stuff/uses some external APIs but there's a point where if I find myself mocking too many things in a test, I take note that the function being tested is probably doing too much at once.

However, I did just the other day hit my first case of a test tricking me into thinking something was OK that wasn't -- I changed the signature of a function, adding a required argument, and failed to add that argument in the one place it got called cause the tests covering the caller mocked the function I changed and made assertions about its call arguments against the old signature; I didn't notice/forgot to change those tests when I changed the signature.

Mocks for external services outside of your control are good. You don't want your test to fail because a remote service is down, or a random network failure. I had one job where everything was mocked in tests and as noted by another poster it made it really hard to change things.

I've come to the point where I lean on end to end testing far more than unit testing. For complicated code I will definitely still unit test, but the code can still be designed such that you do not need to mock anything in most cases.

The March Hare
Oct 15, 2006

Je rêve d'un
Wayne's World 3
Buglord
The smaller your units, the easier the unit tests (also much less likely you need to mock anything that isn't some external API call or whatever).

I worked somewhere that had like literal thousands of lines long chains of if else blocks inside of Django views and those were, more or less, completely loving impossible to test or refactor. If the person(s) writing that code had been following TDD they would have instead written like 30 ~4 line functions to do the same poo poo and it all would have been DRY and testable.

That's an extreme example, and I don't actually like TDD for a number of other reasons, but it can guide people who would otherwise commit atrocities toward, you know, not.

Dominoes
Sep 20, 2007

KICK BAMA KICK posted:

However, I did just the other day hit my first case of a test tricking me into thinking something was OK that wasn't
False positives (in any type of process) are something that's going to happen - it's OK to have some as long as the testing (etc) is a net help.

Good point on the catching all possible errors. I'm still struggling with this in Python; ie I think I've caught the ValueError, but didn't realize that a given API could also throw a different type, that I may not see until an unusual circumstance happens. Any thoughts on how to deal with this? Good API docs help so you can catch each type.

Dominoes fucked around with this message at 09:22 on Jan 24, 2020

Private Speech
Mar 30, 2011

I HAVE EVEN MORE WORTHLESS BEANIE BABIES IN MY COLLECTION THAN I HAVE WORTHLESS POSTS IN THE BEANIE BABY THREAD YET I STILL HAVE THE TEMERITY TO CRITICIZE OTHERS' COLLECTIONS

IF YOU SEE ME TALKING ABOUT BEANIE BABIES, PLEASE TELL ME TO

EAT. SHIT.


Dominoes posted:

False positives (in any type of process) are something that's going to happen - it's OK to have some as long as the testing (etc) is a net help.

Good point on the catching all possible errors. I'm still struggling with this in Python; ie I think I've caught the ValueError, but didn't realize that a given API could also throw a different type, that I may not see until an unusual circumstance happens. Any thoughts on how to deal with this? Good API docs help so you can catch each type.

Yeah this is a really annoying thing about python exceptions. I suppose an IDE can help but even then it's not perfect, I've definitely had it catch me out more than once when using unfamiliar libraries.

Loezi
Dec 18, 2012

Never buy the cheap stuff
Not quite sure this doesn't fit better in the coding horrors thread, but I'm working with a weird system that puts a strong emphasis on processing stuff based on composable pipelines of components. Now, poo poo works correctly from the "running code" point of view, but I'd love to introduce some type checking, maybe even catching some of the most likely bugs ahead of time by, say, running mypy.

I've tried to come up with a minimal working example of what I'm talking about :
Python code:
from abc import ABC, abstractmethod
from typing import List, Tuple, TypeVar, Iterable, Any

WhatDoIPutHere = TypeVar('WhatDoIPutHere')

class Component(ABC):
    @abstractmethod
    def __call__(self, common_a: str, common_b: str, *args: WhatDoIPutHere) -> Tuple[WhatDoIPutHere]:
        ...

class ComponentA(Component):
    def __call__(self, common_a: str, common_b: str) -> Tuple[str]:
        return ("foo",)

class ComponentB(Component):
    def __call__(self, common_a: str, common_b: str, special_a: str) -> Tuple[str]:
        return (special_a + "bar",)

class ComponentC(Component):
    def __call__(self, common_a: str, common_b: str) -> Tuple[int]:
        return (1,)

class ComponentD(Component):
    def __call__(self, common_a: str, common_b: str, special_a: int) -> Tuple[int]:
        return (special_a + 1,)

def run_pipe(components: List[Component]) -> Tuple[Any]:
    args: Iterable[WhatDoIPutHere] = []
    for component in components:
        args = component("farts", "butts", *args)
    print(args)

run_pipe([ComponentA(), ComponentB()]) # Works
run_pipe([ComponentC(), ComponentD()]) # Works
run_pipe([ComponentA(), ComponentD()]) # Breaks, as expected
So the above thing works as I'd expect (outputs are "foobar", "2" and TypeError).

But if anyone has any hints (:haw:) on how to type this monstrosity (or modify it so that it's typeable) I'd really appreciate it. In a perfect world mypy would throw a fit about the TypeError inducing example, but be fine with the other two, but I'd settle for a setup where mypy stops whining about
code:
error: Signature of "__call__" incompatible with supertype "Component"

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Looks like there's a chance that PEP-585 will make it into 3.9. This will make all standard collections able to be used as generic types without special imports from typing.

For example:

Python code:
# instead of this:
def foo(bar: typing.List[int]): ...

# you can do this:
def foo(bar: list[int]): ...

# or instead of this:
def foo(bar: typing.DefaultDict[str, int]): ...

# you can do this:
def foo(bar: defaultdict[str, int]): ...
The special imports from the typing module have always irritated me from a purity and simplicity standpoint.

Hollow Talk
Feb 2, 2014

Thermopyle posted:

Looks like there's a chance that PEP-585 will make it into 3.9. This will make all standard collections able to be used as generic types without special imports from typing.

For example:

Python code:
# instead of this:
def foo(bar: typing.List[int]): ...

# you can do this:
def foo(bar: list[int]): ...

# or instead of this:
def foo(bar: typing.DefaultDict[str, int]): ...

# you can do this:
def foo(bar: defaultdict[str, int]): ...
The special imports from the typing module have always irritated me from a purity and simplicity standpoint.

:getin:

The Fool
Oct 16, 2003


Pandas question:
I have a dataframe that I am doing groupby().agg() on to merge duplicate ID's and sum specific columns.

It has a bunch of columns, but I only need to sum 3 of them.

This all works. My question is, how can I do this without having to specify every single column in the dataframe?

CarForumPoster
Jun 26, 2013

⚡POWER⚡

The Fool posted:

Pandas question:
I have a dataframe that I am doing groupby().agg() on to merge duplicate ID's and sum specific columns.

It has a bunch of columns, but I only need to sum 3 of them.

This all works. My question is, how can I do this without having to specify every single column in the dataframe?

Could you provide an input, expected output and example bit of code and what the output? I'm not sure why you'd specify all column names to accomplish that.

The Fool
Oct 16, 2003


CarForumPoster posted:

Could you provide an input, expected output and example bit of code and what the output? I'm not sure why you'd specify all column names to accomplish that.

Given a table with the following columns that matter, and a bunch of columns that I don't need to process at all:
pre:
 id | a | b | total
----|---|---|--------
 1  | 0 | 3 | 3
 2  | 1 | 0 | 1
 2  | 0 | 3 | 3
 3  | 2 | 0 | 2
 4  | 0 | 2 | 2
I need to merge common id's and sum the columns a, b, and total, the other columns not shown contain metadata about the person represented by the id number and is the same in rows with matching id's

end result looking like this:
pre:
 id | a | b | total
----|---|---|--------
 1  | 0 | 3 | 3
 2  | 1 | 3 | 4
 3  | 2 | 0 | 2
 4  | 0 | 2 | 2
I currently use
code:
table = table.groupby(['ID']).agg({
    'First Name': 'first',
    'Middle Name': 'first',
    'Last Name': 'first',
    'Suffix': 'first',
    'A': 'sum',
    'B': 'sum',
    'Total': 'sum',
    'ID': 'first'
}).reset_index(drop=True)
But I actually have quite a few more columns than that.

My issue is that if I don't explicitly list every single column, they are not present in the resulting dataframe.

vikingstrike
Sep 23, 2007

whats happening, captain
code:
sum_cols = ['colA', 'colB', colC']
agg_dict = {}
for col in frame.columns:
    if col in sum_cols:
        agg_dict[col] = 'sum'
    else:
        agg_dict[col] = 'first'

aggregated = frame.groupby('level1').agg(agg_dict)

The Fool
Oct 16, 2003


That'll work a treat, thanks.

Didn't occur to me to built the dict like that, I was assuming pandas would have some flag somewhere that would do it automatically.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
Has anybody ever seen a situation where the function a thread is running completely finishes and clears, but a join on it hangs? If it adds an interesting wrinkle, then the thread has a Queue and it thinks its empty when it checks on the way out.

Edit: Apparently I was double-launching the thread as a consequence of a change I had made. I need a guard on that kind of thing.

Rocko Bonaparte fucked around with this message at 08:26 on Jan 30, 2020

Csixtyfour
Jan 14, 2004
Taking a python class, whats a better way to do this? My way works, but just seems clunky.

19. Write a Python program that requests five integer values from the user. It then prints one of two things:
if any of the values entered are duplicates, it prints "DUPLICATES"; otherwise, it prints "ALL UNIQUE".

code:
f = int(input("Please enter a number "))
g = int(input("Please enter another number "))
h = int(input("Please enter another number "))
i = int(input("Please enter another number "))
j = int(input("Please enter another number "))

if f == g or f == h or f == i or f == j or g == h or g == i \
   or g == j or h == i or h == j or i == j:
 print("DUPLICATES")

else: print("ALL UNIQUE")

taqueso
Mar 8, 2004


:911:
:wookie: :thermidor: :wookie:
:dehumanize:

:pirate::hf::tinfoil:

Trying not to spoil it completely: you want to make a loop that asks for numbers, and instead of recording each number in its own variable, you want to keep track of them in a dictionary or set or list. That way you can handle any number of inputs.

You could check at the end or you can check each time a new number is added.

e: Oh, the assignment does say it will always be five numbers. Using a loop is still a good idea, so you don't have to write the logic five times.

e2: if you haven't learned about loops and sets/dicts yet then your solution is probably what they expect to see

taqueso fucked around with this message at 03:18 on Feb 3, 2020

KICK BAMA KICK
Mar 2, 2009

A set is the basic data structure oriented around the idea of uniqueness, so there's a couple of ways you could use that in your solution. And if you're putting the values in a container, rather than storing them with individually-named variables, you can more easily loop (over a range(5), maybe?) to get your inputs.

e:fb

NinpoEspiritoSanto
Oct 22, 2013




Apologies I am phone posting so no code but I'd add the integer inputs to a list as many times as needed in a while loop. Then I'd check len of that list against the output of the len of that list converted to a set. If same length, all unique, else duplicates.

e: soundly beaten.

porksmash
Sep 30, 2008
Look into lists and loops.

edit: put my rear end in line I guess

Csixtyfour
Jan 14, 2004
Thanks for lists and loops, Ill post what I come up with tomorrow. Turned in my home work with my posted code, was just curious of the better way to do it.

taqueso
Mar 8, 2004


:911:
:wookie: :thermidor: :wookie:
:dehumanize:

:pirate::hf::tinfoil:

This assignment is a perfect setup for "OK, great but what if they want to enter 3 numbers or 7 numbers or 25? How do we accept numbers until the user tells us they are finished?" "Well, check out this thing called a loop." as the next class.

Sad Panda
Sep 22, 2004

I'm a Sad Panda.
Just one thing no one mentioned, num1, num2, num3, num4, num5 would be much better variable names.

nitsuga
Jan 1, 2007

At work we've got some Python scripts we've written to help us with various tasks. A number of them relate to one another, and import one as a module frequently (we pull data then put them into tickets, et cetera). None of us are proper developers, and we've had a fair amount of trouble getting people setup with the tools.

Without getting too E/N, what's the best way to structure these scripts in a repo? I'd love to be able to have people clone the repo, set up a virtual environment, and install the requirements and go. Is that a terrible way to do it? I figure this way we can just update the repo on our computers when we make minor changes rather than our current system which relies on them being in specific directories.

For clarity, here's a diagram of what I'm thinking:

nitsuga fucked around with this message at 19:36 on Feb 3, 2020

mbt
Aug 13, 2012

how often are you setting this up and how often are you making updates to the scripts?

nitsuga posted:

None of us are proper developers,
ah yes the old "no true developer" fallacy

Adbot
ADBOT LOVES YOU

nitsuga
Jan 1, 2007

Meyers-Briggs Testicle posted:

how often are you setting this up and how often are you making updates to the scripts?

ah yes the old "no true developer" fallacy

I think I know what you're getting at, how about the best of us are still pretty junior? As for updates, it isn't all that frequent really. In any given week we could have one or two minor updates. Sometimes more sometimes less.

nitsuga fucked around with this message at 20:15 on Feb 3, 2020

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply