cinci zoo sniper posted:

If your dataset has only 10000 records for test then this will be off by one error, maximum i is 9999 and can not be equal 10000, and the last modulo operation will resolve to not True.

bob dobbs is dead posted:

itertools is one of the great parts of python, only they have a very significant and good as hell portion of it which is not actually part of the library but is shown as "itertool recipes"

you want this one in it

code:

def grouper(iterable, n, fillvalue=None):
    "Collect data into fixed-length chunks or blocks"
    # grouper('ABCDEFG', 3, 'x') --> ABC DEF Gxx"
    args = [iter(iterable)] * n
    return zip_longest(*args, fillvalue=fillvalue)

SurgicalOntologist posted:

And if you haven't seen it before, tqmd is just a progress bar library (a pretty great one)

SurgicalOntologist posted:

Oh, figured it out. https://github.com/tqdm/tqdm/issues/613

bob dobbs is dead posted:

itertools is one of the great parts of python, only they have a very significant and good as hell portion of it which is not actually part of the library but is shown as "itertool recipes"

Thermopyle posted:

This has always made me laugh.

"Let's write a bunch of useful and helpful code. Then lets take it and put it in the documentation instead of making it actually importable."

salisbury shake posted:

I'm just learning to use NumPy and had to poke around the docs to do what I want. While the above code finds the solution correctly, I'm unsure if I'm using NumPy efficiently or even canonically.

For example, in get_fabric(), I'm trying to minimize sequential Python code in favor of using NumPy's ufuncs and routines where possible, but I'm still calling np.add.at() over a thousand times in total. That feels wrong, but I don't have the context to really know if/why it's wrong.

salisbury shake posted:

I'm just learning to use NumPy and had to poke around the docs to do what I want. While the above code finds the solution correctly, I'm unsure if I'm using NumPy efficiently or even canonically.

For example, in get_fabric(), I'm trying to minimize sequential Python code in favor of using NumPy's ufuncs and routines where possible, but I'm still calling np.add.at() over a thousand times in total. That feels wrong, but I don't have the context to really know if/why it's wrong.

CarForumPoster posted:

I got another easy/dumb one.

Have df["Time"] with

code:

2 days
NULL
NULL
3 yrs
NULL
2 wks
1 yrs1 day

I want to convert these values to days where NULL = 0. I do this once a week on about 2k rows.

code:

temp = df["Time"].str.split()
table = []

for i in range(len(temp)):
    if temp[i][1] == "days":
        temp[i][0] = int(temp[i][0]) 
    elif pd.isnull(temp[i]):
        temp[i] = [0,0]
    elif temp[i][1] == "yrs":
        temp[i][0] = int(temp[i][0])*365
    elif temp[i][1] == "mths":
        temp[i][0] = int(telp[i][0])*30
    elif temp[i][1] == "wks":
        temp[i][0] = int(temp[i][0])*7
    elif temp[i][1] == "yrs1" and temp[i][2] == "days":
        temp[i][0] = int(temp[i][0])*365 + 1
    else:
        temp[i][0] = "some other regex"
    table.append([temp[i][0]])
    print([temp[i]])

df["TimeDays"] = pd.Series(table)

This throws the following when it gets to NULL:
4 for i in range(len(temp)):
----> 5 if temp[i][1] == "days":
6 temp[i][0] = int(temp[i][0])
7 elif pd.isnull(temp[i]):

TypeError: 'float' object is not subscriptable

But I think the real issue is the "isnull" part.

I think its telling me hey there is no "ith" element to look up...but there is right?

cinci zoo sniper posted:

It tries to get 2nd element inside NULL, that temp[i] corresponds to. Also I would suggest to use dateparser for this.

bob dobbs is dead posted:

eucliean distance metric is fine unless you're using the map for navigation tho?

2-space is within the set of n-spaces, yes

what do the latitude and longitude data points look like?

Jose Cuervo posted:

From the Python Data Science Handbook it seemed to say that you should use 'haversine' when performing KDE where the points are latitude and longitude - that is why I went from using 'euclidean' to 'haversine'. By 'fine' do you mean the error in the distances between points (because the distances will not be great circle distances but just straight line distances) is small enough to be ignored if you are not trying to navigate between points?

I am not sure what the '2-space is within the set of n-spaces, yes' comment means exactly.

The Longitude minimum and maximum are -83.6311 and -75.3771, while the Latitude maximum and minimum are 36.5454 and 39.4172. I believe they are in units of decimal degrees. Is this what you were asking?

bob dobbs is dead posted:

what i thought that you were thinking is that "oh, you can't use euclidean at all for this", but the real statement to make is "euclidean will introduce distortions in this. but if you're overlaying it over a flat projection in the first place, the distortions will basically look like the map distortions". the other possible misunderstanding is just because euclidean distance is viable for any dimensional space (n-spaces) you think you can't just use it for 2-dimensional space (2-space)

also, the underlaying mistake is prolly you need it radians

SurgicalOntologist posted:

I'm doing a bunch of computations and saving them to a database. The following function is supposed to be committing to the database every 100 results (based on the variable commit_interval), and stopping after 10,000 (the value of total). But I'm not getting 10,000 results on each backtest_set, instead I'm getting 9,900. Can anyone spot the error? Am I making a basic off-by-one -ish mistake or do I have some weird race condition? I can't figure it out. Here's the code.

Python code:

    results = []
    for i, result in enumerate(tqdm(
            backtest_set.run(),
            unit='entry', desc='Backtesting', total=total, unit_scale=True,
    ), 1):
        results.append(dict(backtest_id=backtest_set.id, fantasy_points_hundredths=result))
        if not i % commit_interval or i == total:
            db.execute(models.BacktestResult.__table__.insert(), results)
            db.commit()
            results = []
        if i == total:
            break

Any ideas?

The actual work is here is done inside the generator function backtest_set.run(). And if you haven't seen it before, tqmd is just a progress bar library (a pretty great one), as I'm using it just wraps an iterable. The only other things there are my models and sqlalchemy session db.

School of How posted:

Does anybody know of a good python library that has an implementation of "nth prime" that works in both python 2 and 3?

>>> from some_library import nth_prime
>>> nth_prime(999)
7907

You'd think this would be easy to find, but all I can find from googling are snippets on stackoverflow that only work in python 2.

bob dobbs is dead posted:

thats cuz your search keyword is "sieve of erastothenes" or "number sieve", not "nth prime"

its not precisely what you want but its real close

School of How posted:

Thats a completely different algorithm. Sieve gives you primes below a certain number, thats different than nth prime...

School of How posted:

Does anybody know of a good python library that has an implementation of "nth prime" that works in both python 2 and 3?

>>> from some_library import nth_prime
>>> nth_prime(999)
7907

You'd think this would be easy to find, but all I can find from googling are snippets on stackoverflow that only work in python 2.

QuarkJets posted:

Not using self means that you've defined a class attribute. This is valid, all instances of the class are accessing a shared version of the variable and that may be the behavior that you want. foo.num and bar.num will always be the same value in your code

Using self creates an instance attribute, meaning all instances of the classes are accessing their own version of the variable. Declaring "self.num = 0" in your constructor creates an instance attribute, and then foo.num and bar.num can be different values

Gothmog1065 posted:

Ah, okay. That actually explains the variable instance better. For some reason I thought if defined outside of the __init__, it was a class variable, while inside the __init__ it was an instance. That actually helps clear up a lot, thank you!

The XKCD Larper posted:

A question about class variables. I watched the MIT OCW video about class objects and the instructor says that class variables should be set, and retrieved, using getter and setter functions as opposed to directly assigning them. They say that this allows for renaming variables within the class and making renaming variables easier. Is there any other reason that one would work this way?

The XKCD Larper posted:

A question about class variables. I watched the MIT OCW video about class objects and the instructor says that class variables should be set, and retrieved, using getter and setter functions as opposed to directly assigning them. They say that this allows for renaming variables within the class and making renaming variables easier. Is there any other reason that one would work this way?

The XKCD Larper posted:

A question about class variables. I watched the MIT OCW video about class objects and the instructor says that class variables should be set, and retrieved, using getter and setter functions as opposed to directly assigning them. They say that this allows for renaming variables within the class and making renaming variables easier. Is there any other reason that one would work this way?

bob dobbs is dead posted:

dont learn oop

Kilometres Davis posted:

keyframe posted:

I am getting my feet wet with web scraping. Is there a reason to use beautifulsoup over selenium? I find selenium much easier to work with but I am doing super basic stuff right now while learning. Was just curious if BS brings something to the table selenium doesn't have.