Lexical Unit posted:

Python code:

In [1]: a = ['a', 'b', 'c', 0, 'd', 'e', 'f', 0, 'g', 'h', 'i']

In [2]: b = [a for a in a if a != 0]

In [3]: b
Out[3]: ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i']

Like that?

accipter posted:

Instead of telling the GUI to update, emit a signal from the model/data class. On the GUI side of things, connect to the signal of the model/data class and update the GUI as needed.

Pollyanna posted:

Say I have a list like [a, b, c, 0, d, e, f, 0, g, h, i]. How do I extract all elements between instances of 0?

Pollyanna posted:

Say I have a list like [a, b, c, 0, d, e, f, 0, g, h, i]. How do I extract all elements between instances of 0?

Thermopyle posted:

Python code:

In [2]: a = ['a', 'b', 'c', 0, 'd', 'e', 'f', 0, 'g', 'h', 'i']

In [3]: a[a.index(0) + 1 : a[a.index(0) + 1:].index(0)+a.index(0)+1]
Out[3]: ['d', 'e', 'f']

Don't do this, as it's inefficient, but I just felt like doing a one liner.

Pollyanna posted:

and what I need is the string between both <hr/> tags. In your example, I'm looking for [d, e, f].

Or maybe there's an easier way to do this, does anyone know Beautiful Soup

FoiledAgain posted:

Supposing the first element fails the test, the program still has to access that list 5001 time before realizing that, right? It appends 5000 elements to the list first, passes that list to all(), checks 1 element and returns False. Worst case, I have to append 5000 times and then check 5000 times.

If I do this with a generator, I can avoid this problem. I think. Suppose the first item fails the test. all((test(x) for x in X)) would create one generator object, not a list of 5000 items, pass that generator to all(),and then return false after only checking 1 thing. Worst case it only checks 5000 things.

Am I understanding correctly?

Pollyanna posted:

Say I have a list like [a, b, c, 0, d, e, f, 0, g, h, i]. How do I extract all elements between instances of 0?

suffix posted:

Start at the first <hr/> tag and keep going until you hit the other?
e.g.

Python code:

def element_is_not_hr(element):
    return element.name != 'hr'

important_stuff = itertools.takewhile(element_is_not_hr, document.hr.next_siblings)

Pollyanna posted:

edit: I think takewhile does something different than what I need, it looks like the way it's written right now it takes everything that is NOT <hr/>, which is not quite what I need. I need it to not take anything until it FINDS an <hr/>, and then stop once it sees an <hr/> again... lemme mull over this some more.

breaks posted:

Yes, you have got the idea. If you use a list comprehension it must determine the value of every element in the list before it does anything with it. A generator determines its values as it goes, so there is no need to figure it all out in advances, which is advantageous if figuring it out takes a while.

Pollyanna posted:

I'll start trying to get the text directly from the elements for now, before I try and tackle a string.


Thing is that I don't think BS takes "document.hr" as valid, at least not when I've tried to do so.

I know that contents outputs a list, so I tried using this on that list. Unfortunately, it gives me this:

Python code:

<itertools.takewhile object at 0x10163e560>

What is this?

edit: I think takewhile does something different than what I need, it looks like the way it's written right now it takes everything that is NOT <hr/>, which is not quite what I need. I need it to not take anything until it FINDS an <hr/>, and then stop once it sees an <hr/> again... lemme mull over this some more.

edit2: i swear i'm missing something really obvious here

SirPablo posted:

am I missing a quicker way of doing this?

Lysidas posted:

Yes. Tar files don't provide random access to their contents; to get at a certain file you have to read the tar file up to its position. This can be optimized by seeking from one tar entry to the next, but you're still taking O(n²) time to do something that can be done in O(n). You should iterate over the TarInfo with TarFile.next() and extract each entry if it matches your criteria.

Nippashish posted:

I can't tell what proportion of the lines have TMAX in them, but if you're ignoring most of the them it will probably be a lot faster to select those lines outside of python with grep or similar and create files that have only the lines you want for python. The speed difference between grepping 2.4GB of text and looping line by line over 2.4GB of text in python is quite large. This might help even if you are using most of the lines, since it would avoid doing two passes through each file in python.

Philip Rivers posted:

So is there a goon-consensus preferred introduction to Python, or are any of them in the OP good? I'm in a situation where I need to learn the basics of Python as quick as I can, but I don't have any access to formal instruction. I'm self motivated, but I have attention issues, so if the reading is too slow, I'll get bored, and if it's too fast, I'll probably just lose focus.

I really wish the CS department at my school wasn't so impacted. I'd just take the intro to Python class if I could, but that's not really an option.

Philip Rivers posted:

So is there a goon-consensus preferred introduction to Python, or are any of them in the OP good? I'm in a situation where I need to learn the basics of Python as quick as I can, but I don't have any access to formal instruction. I'm self motivated, but I have attention issues, so if the reading is too slow, I'll get bored, and if it's too fast, I'll probably just lose focus.

I really wish the CS department at my school wasn't so impacted. I'd just take the intro to Python class if I could, but that's not really an option.

digitalcamo posted:

Had a question about functions. Codecademy has me do program challenges and sometimes it will say make an argument called something_one that takes something as an argument. Other times it will ask to take something as input. I keep getting confused about the two. Could someone explain the difference between arguments and input. I thought arguments and input were what you put in (), like in something_one(input/argument).

digitalcamo posted:

Had a question about functions. Codecademy has me do program challenges and sometimes it will say make an argument called something_one that takes something as an argument. Other times it will ask to take something as input. I keep getting confused about the two. Could someone explain the difference between arguments and input. I thought arguments and input were what you put in (), like in something_one(input/argument).

Ireland Sucks posted:

Is it possible to read from a file on Windows while another process is writing to it? I'm trying to process logfile updates as fast as they are written but whenever I read some data the writer process complains about the file being used by another process and dies.

e:did it by checking for a lock and then reading. race conditions ahoy!

keanu posted:

If you know how to program already then Dive Into Python might work for you.

Philip Rivers posted:

Dominoes posted:

Scipy question for anyone familiar with Find_peaks_CWT or Scipy signal analysis.

I'm writing an instrument tuner. I'm trying a technique that uses a hybrid of FFT and calculating peak widths.

1: FFT precision seems to be equal to sample rate/size. For example, (44100hz rate / 1024 sample size) = precision of 43hz. With this in mind, a large sample size is required for acceptable precision, especially at low frequencies to to the logarithmic scaling of pitch. Ie to get 1hz precision, you need 1 second of data, meaning slow, choppy updates.

2: I tried a basic windowing method involving a long sample that adds new chunks to the end and slices chunk off at the beginning. This doesn't solve the problem: It just replaces slow analysis/update rates with lagged response. I don't think a smarter windowing method would solve this.

3: Calculating the width between peaks, then diving sample rate by it gives a precise result. (Ie: 44100hz / 100 sample peak distance) = 441hz signal. However, it requires a smooth signal without harmonics, is quirky, and I don't fully understand how the Scipy peak finding algorithm works. The docs are barebones.

4: I'm trying a hybrid approach. Use a low-sample size FFT (fast, imprecise), take the maximum FFT response, and feed that into a peak-finding algorithm. Set a low-pass filter based on the FFT-calculated freq to make the signal smooth and easy for the peak finder to understand. Set expected peak width based on the fft-calculated freq. Average however many peak widths are in the sample, and throw out outliers.

Example raw and low-passed signal of a A-440hz piano sample. You could calculate a frequency by taking average width between peaks of the low-passed signal, or by taking the total distance between start and end peaks, and dividing it by the number of peaks.



Main question: How should I set the peaks width? The input requires an array. I'd expect it to take min and max value, but instead it seems to want many values; examples online show using a range. I can figure out a range of peak values that may work: Ie: the width could be 1/4 of the fft-calculated freq. How should I set up the array? I can narrow down the expected peak width pretty closely given I have a nearby frequency, and and no harmonics. Should the width array just be one value? Can anyone explain how to use the other, optional parameters?

accipter posted:

Regarding the Fourier transform. The time step (dt) of the signal defines the maximum frequency (1/(2 dt)) -- which is called the Nyquist frequency. The duration of signal defines the lowest frequency.

Since you mentioned that you are doing an instrument tuner, it sounds like you want to find the dominant frequency of the signal. If you plot the Fourier Amplitude spectrum (FAS), the peak in the signal corresponds to the peak in the FAS. So rather than trying to find peaks in the time series. Take the FFT, smooth the FAS, and find the peak.

Dominoes posted:

Scipy question for anyone familiar with Find_peaks_CWT or Scipy signal analysis.

Dominoes posted:

Here's a similar example with FFT of a 440hz piano sound plotted:


If I understand correctly, what you're describing is the method I use to get the reference FFT frequency I use to set the peak finder's parameters. The problem I'm encountering with FFT is that the peak in the subplot3 is 430.7hz instead of 440hz, due to its 44100hz/1024=43hz precision. The only output it can give is in increments of 43hz, with this sample rate and size. The low pass filter that generates the subplot2 signal is based on this 430hz reference freq.

The peak finder gives 100.22 sample average peak spacing on the subplot2 signal in the chart. Dividing 44100hz (sample rate) by this gives 440.02hz. Unfortunately, the peak finder is wonky and doesn't always act as expected. I think I can fix this by changing its parameters, which I don't know how to do.

Modern Pragmatist posted:

Why not upsample the original signal so that you have more than 1024 samples in your window and therefore better frequency resolution?

quote:

Also be careful where your 0 point is in your power spectrum. Zero should be the DC point and therefore be the largest peak in the spectrum. The first point in your spectrum is not the DC point but the one just outside of it.

Dominoes posted:

That sounds like point 2 in my original post: It causes the response to be delayed when the frequency changes. 1 cent precision is ideal. cents = 1200 * math.log2(measured_freq/note_freq) Feeding 441 and 440 in as the measured and note freqs, 1cent near 440hz = 4hz. Given the precision formula I observed, You'd need a 4-second sample to get 1 cent precision at A440 using FFTs alone. Longer at lower freqs, shorter at higher.

Thanks - I'll read about and fix that. It's currently the scipy default, with the negative side removed.

Dominoes posted:

Main question: How should I set the peaks width? The input requires an array. I'd expect it to take min and max value, but instead it seems to want many values; examples online show using a range. I can figure out a range of peak values that may work: Ie: the width could be 1/4 of the fft-calculated freq. How should I set up the array? I can narrow down the expected peak width pretty closely given I have a nearby frequency, and and no harmonics. Should the width array just be one value? Can anyone explain how to use the other, optional parameters?

evensevenone posted:

Up sampling wont work. The data isn't there. A 440hz sine sampled at 44khz for 1000 samples won't actually look any different than a 430hz signal, at least not in a way that can lead you to discriminate the two.

Casyl posted:

I've messed around with scipy's implementation of cwt a bit, but it's a tough subject to get around so I didn't make all that much progress, but I'll share what I've figured out. scipy.signal.cwt takes your input vector and does the wavelet transform using each width, returning an array where each element is the smoothed vector at one width. So if your widths were [1, 2, 3, 4, 5] then your returned cwt array would have 5 entries, the first being the wavelet transform of the input vector using width = 1, the second being the cwt with width = 2, and so on. From what I can gather, when you use signal.find_peaks_cwt, the algorithm somehow looks at all of these wavelet transforms to determine if a peak is there or not (through "ridges" and whatever else it talks about), and returns an array of peak values. The find_peaks_cwt page references this paper that goes into more depth about how it picks the peaks, but it's really dense. If you can get through it it may help you get a sense of what the other parameters do.

tzec posted:

That isn't true; there's plenty to discriminate them (as a quick test, you can hear the difference with your ears even on a 1000 sample burst). Upsampling is a very inefficient way to do improve things, though.

Dominoes, if your signals are going to be clean like your examples, you might try simply bandpassing the signal and then counting the zero crossings (no need to use a peak finder). You can do interpolation to get sub-sample accuracy if you need to. If you want, you can try estimating the bandpass center frequency from the DFT peak.

Or you could use the autocorrelation (with a sliding window), which is a standard way of estimating the fundamental (e.g. see http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.72.1676&rep=rep1&type=pdf for a mathematical overview).

Alternatively, this paper http://courses.physics.illinois.edu/phys406/NSF_REU_Reports/2005_reu/Real-Time_Time-Domain_Pitch_Tracking_Using_Wavelets.pdf shows a simple to implement and efficient wavelet pitch tracking algorithm (there's a C implementation at http://www.schmittmachine.com/dywapitchtrack.html )

fritz posted:

Go read something like this: http://www.wpi.edu/Pubs/ETD/Available/etd-042511-144644/unrestricted/YLiao.pdf starting at about page 13. Or this: http://www.mathworks.com/help/signal/ref/rootmusic.html. Or do an autocorrelation like tzec suggested. Are you insisting on the fourier spectrum for non-technical reasons?

Hammerite posted:

In Python it's generally recommended to deal with things like this using exceptions. So instead of

code:

if file exists and is not being used:
   do things with file
else:
   complain to user or whatever

you do

code:

try:
   do things with file
except IOError:
   complain to user or whatever

This is often summarised by the saying "it is easier to ask forgiveness than permission" (EAFP).

BigRedDot posted:

Whether you should "look before you leap" or "ask for forgiveness" depends on how often you expect the "fail" case to occur. For instance if you put a try-except in a loop that will fail regularly even 10% of the time you can incur a performance penalty upwards of 30%.