Dominoes posted:

Allow for Python function annotations.

Method one: Ignore them, like the python interpreter does, instead of throwing an error. Should be easy to implement.

Method two: Use them as an alternative syntax for function signatures, like with mypy.

SurgicalOntologist posted:

I just wanted to point out that it is possible to compute variance in a single pass.

BigRedDot posted:

I think the main issue is that the mypy's type annotations are not generic or extensible enough (currently) to accommodate all the cases numba handles, so there is not alot of push in that direction. Maybe someday, as a subset, if there is enough request for it.

jimcunningham posted:

Still a programming newbie. Would like some help with enumerate(). When I try to learn stuff on my own, it's hard to follow because of the different variables they are using.
I'm reading a txt file into a list. I need to number each line. Then print even numbered lines. I made it work like this, but I have a feeling that enumerate works better. ( I discovered it afterwards)

code:

or_file = open("test5.txt", 'r')
outtext = or_file.readlines()
or_file.close()
f_text = list()
x = 1
newtxt = list()

for line in outtext:
	y = str(x)
	newtxt.append(y + " " + line)
	x = x + 1

for line in newtxt:
	f = int(line[0])
	
	if (f % 2 == 0):
		f_text.append(line[:])
		
	else: continue
	
print f_text

quote:

I can probably get rid of 10 lines or more by doing 1 crazy trick. and I bet it's simple and everyone is laughing at me.

Dominoes posted:

Should still be able to ignore them. (Not just mypy - any valid Python func annotation).

Another q - might be more of a C / memory allocation question. What's the best way to trim empty values from an array?

I'm making a substitute for np.nonzero:

Python code:

   result_i = 0
    result = np.empty(M, dtype=np.int)
    for i in range(M):
        if diffs[i] != 0:
            result[result_i] = i
            result_i += 1
    return result

I'm setting up the array to its maximum size. I get a valid (and faster) result than np.nonzero, but it has a bunch of 0s and arbitrary numbers hanging off the end. Is there a fast way to trim it, or a better way to approach? Initializing the array with 0s and using np.trim_zeros is far too slow. I don't expect much/any speed increase over numpy's func, but I'm curious academically.

I tried this:

Python code:

    result_i = 0  # index, and also serves as size for new array.
    result = np.empty(M, dtype=np.int)
    for i in range(M):
        if diffs[i] != 0:
            result[result_i] = i
            result_i += 1

    result_trimmed = np.empty(result_i, dtype=np.int)
    for i in range(result_i):
        result_trimmed[i] = result[i]

    return result_trimmed

, but it ends up taking longer than np.nonzero.

Related question to Dot: Do you know why using np.empty, np.zeros etc will cause Numba to fail when using nopython=True?

Python code:

@numba.jit(nopython=True)
def test():
    np.empty(50)
    return 2


...

UntypedAttributeError: Failed at nopython (nopython frontend)
Unknown attribute "empty" of type Module(<module 'numpy' from 
'C:\\Users\\David\\Anaconda3\\lib\\site-packages\\numpy\\__init__.py'>)

Dominoes posted:

That wikipedia function for single-pass variance , with an added numba decorator, works twice as fast as the two-pass method I was using. (Pass 1 is finding the mean)

accipter posted:

The next two moments can also be included in one pass.

Dominoes posted:

Faster GLM/RLM would be nice though; coincidentally it's the limfac in one of my projects.

Cingulate posted:

Have you looked at MATLAB's mldivide?

dantheman650 posted:

I can't for the life of my understand why the return value is None when it correctly calculates and prints the sum I want to return inside the same if block! Please come to my rescue again, wonderful Python goons.

Dominoes posted:

Prepend return to the function's last line.

Dominoes posted:

Prepend return to the function's last line.

Is there a reason why python doesn't implement numpy/matlab-style indexing with multiple keys or indices?

Ie for a list of dicts:

Python code:

adict[3, 'akey']

As a cleaner alternative to

Python code:

adict[3]['akey']

QuarkJets posted:

This is the first time that I've ever heard someone suggest MATLAB in response to "I need this to run faster"

Dominoes posted:

Prepend return to the function's last line.

Is there a reason why python doesn't implement numpy/matlab-style indexing with multiple keys or indices?

Ie for a list of dicts:

Python code:

adict[3, 'akey']

As a cleaner alternative to

Python code:

adict[3]['akey']

Dominoes posted:

Is there a reason why python doesn't implement numpy/matlab-style indexing with multiple keys or indices?

Ie for a list of dicts:

Python code:

adict[3, 'akey']

As a cleaner alternative to

Python code:

adict[3]['akey']

Cingulate posted:

of course, many of MATLAB's basic capabilities are state of the art, top of the line.

QuarkJets posted:

This is a dirty lie (unless you're choosing a very narrow definition of "basic") and you shouldn't spread it

Cingulate posted:

mldivide is a very basic capacity of MATLAB - it's a single-character operator, more basic than element-wise matrix operations! - and for all I know, state of the art, top of the line. I don't know what you're talking about.

jimcunningham posted:

KICK BAMA KICK posted:

Is there any whitespace in the input file other than the newlines? If not, why all the split()ing and .strip()ing? As I understand it, what you intend to do is take lines in groups of two, the first line of each pair being the header from which you need to extract that four-digit (they're always four digits?) ID and the second being the DNA string to process. I really can't make heads or tails of what's happening in that comprehension to tell you exactly what the IndexError is but I'm pretty sure at least one of the problems is just because you're processing each line of the file rather than considering headers and DNA strings differently. Here's the approach I'd take:

• Read the file into memory with .readlines() so we can access its contents as a list of lines
• Don't build a dict unless you need to do more calculations -- just set variables like max_score and id_of_max and keep track as we iterate over the file
• Iterate over indices stepping by twos to consider a pair of lines at a time, i.e., for i in range(0, len(lines), 2):; then line[i] is the header, from which you can easily extract the last four characters for the id and line[i+1] is the DNA string
• Calculate the score of each DNA string and update your max variables if necessary; at the end of the iteration, those hold your answers

QuarkJets posted:

I'm talking about all of the other "basic features" of MATLAB that are notoriously slow and archaic as gently caress. The features of Matlab that were inherited from older projects work great. Basically, any feature that can't be vectorized with a function written in Fortran before 1990 is complete garbage. Mathworks can't even provide basic list functionality without making it an O(n) operation, for gently caress's sake

By your logic, Fortran77 is a state of the art programming language

jimcunningham posted:

No whitespace. Like i said, I didn't write that line I was having trouble shooting poo poo into the dict. got help on SO.
strip really doesn't need to be in there, but I just found out that they were seeing it as a different format so strip was useful.

the key needs to be after > so Rosalind_3092. Always 4 numbers.
I did kind of what you are saying to do without a dict, but still couldn't do it. I thought this would be easier. I was wrong. I'm gonna learn more basics before I do anything else.

Cingulate posted:

I think you're trying to disprove something quite different from what I stated, such as "MATLAB is a good general programming language" or something like that.

mldivide is a basic MATLAB functionality, and it is state of the art, top of the line. I assume the same goes for e.g. dot products or matrix inversions. Thus, many of MATLAB's basic capabilities are state of the art, top of the line.

QuarkJets posted:

And I agreed with you, MATLAB's features that were built pre-1990 in Fortran by someone other than Mathworks run really well, so long as you can completely vectorize the operation

Cingulate posted:

UMFPACK was released in 1994 and is in C though, and I assume the parts of mldivide that check if UMFPACK is appropriate are not written in pre-1990 Fortran either.
And I guess this must be my final contribution to this slightly silly derail.

SurgicalOntologist posted:

Well, in the end the hard work is done by LAPACK or MKL or some other low-level implementation. So, it is probably not appreciably different in efficiency from any other high-level language which also calls out to those libraries.

http://stackoverflow.com/a/18553768