Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

LochNessMonster: Feb 3, 2005; I need about three fitty

BigRedDot posted:

Let me know if I can answer any questions

I will, thanks!

# ? Oct 17, 2016 18:42

Adbot: ADBOT LOVES YOU

# ? May 8, 2024 10:07

Dr Monkeysee: Oct 11, 2002; just a fox like a hundred thousand others; Nap Ghost

Xarn posted:

Need help with boto3 again .

Is there a way to bypass JSON deserialization of incoming CloudWatch data? Having them in dict is nice and all, but one of my use cases is literally to write JSON to a file (that is then sent somewhere), and having the data flow go
code:
json from server -> boto3 deserialization -> python dict to my application -> dict to json serialization -> write json to gzipped file
gets in my grill (and costs me money, because its running inside of lambda).

How much is it costing you? I can't imagine the deserializing/serializing loop takes any noticeable amount of time compared to the vast cost of talking to CloudWatch and writing to persistent store.

edit: it looks to me like boto doesn't allow you to do this but all AWS services are ultimately just REST endpoints so you could always do it yourself with requests and don't deserialize the response. I'd be very surprised if this was actually a problem though.

# ? Oct 17, 2016 23:33

Dr Monkeysee: Oct 11, 2002; just a fox like a hundred thousand others; Nap Ghost

LochNessMonster posted:

So now I've built my scraper/parser and put the data in a sqlite3 db, but now comes the presentation.

What's the easiest way to make a website do queries on the db to show the data. Can I do that with python too, or should I look into JavaScript for that?

If you really wanted to expand your tech exposure you could expose your db as an HTTP web service (in which case I'd recommend Flask and the plugin Flask-RESTful) and build the website client-side with React or Angular or something. I wouldn't necessarily recommend this over a server-side web site but it's a different approach that is widely used in industry today.

# ? Oct 17, 2016 23:38

Eela6: May 25, 2007; Shredded Hen

Tons of problems involve people importing data from a CSV where the first row is the LABELS of the CSV.

I always find myself converting these labels into namedtuples so I can access the CSV's info either positionally or by keyword.

So I wrote some small helper functions to easily read and write labeled csvs.

This is not high-level python, but hopefully it's useful or interesting to someone!

# ? Oct 18, 2016 07:28

LochNessMonster: Feb 3, 2005; I need about three fitty

Dr Monkeysee posted:

If you really wanted to expand your tech exposure you could expose your db as an HTTP web service (in which case I'd recommend Flask and the plugin Flask-RESTful) and build the website client-side with React or Angular or something. I wouldn't necessarily recommend this over a server-side web site but it's a different approach that is widely used in industry today.

I was considering this when looking into the differences between django and flask. And making each functionality a microservice (sorry for buzzwords) actually sounds like a great idea since I have no clue what I'll be doing next. It's just a hobby project I started and keep expanding. Making components small and (relatively) easy to change sounds like a good idea. That said, I'll first try and get a default flask application running. After that I'm gonna experiment with a RESTful api. When I'm ready I think I'll need to head over to the front end thread and start a religious war by asking whether to use React or Angular.

# ? Oct 18, 2016 08:07

Dex: May 26, 2006; Quintuple x!!!

Would not escrow again.

VERY MISLEADING!

LochNessMonster posted:

I was considering this when looking into the differences between django and flask. And making each functionality a microservice (sorry for buzzwords) actually sounds like a great idea since I have no clue what I'll be doing next. It's just a hobby project I started and keep expanding. Making components small and (relatively) easy to change sounds like a good idea. That said, I'll first try and get a default flask application running. After that I'm gonna experiment with a RESTful api. When I'm ready I think I'll need to head over to the front end thread and start a religious war by asking whether to use React or Angular.

falcon is pretty handy for rest apis

# ? Oct 18, 2016 08:18

BedBuglet: Jan 13, 2016; Snippet of poetry or some shit

Can someone explain to me why Python uses a mutex (the GIL) when you're green threading? Like, the best I can guess is that maybe the cpython has a risk of creating race conditions due to threads of cpython running faster than Python code in the process?

Also, is Python itself already green threaded? It looks like, when you make a user thread in Python, you're actually threading into Python's underlying threading.

I don't know, the poo poo confuses me. It's some next level inception bs. All I know is that Python seems to get increasingly slower the more I use threading.

# ? Oct 19, 2016 11:10

QuarkJets: Sep 8, 2008

BedBuglet posted:

Can someone explain to me why Python uses a mutex (the GIL) when you're green threading? Like, the best I can guess is that maybe the cpython has a risk of creating race conditions due to threads of cpython running faster than Python code in the process?

Also, is Python itself already green threaded? It looks like, when you make a user thread in Python, you're actually threading into Python's underlying threading.

I don't know, the poo poo confuses me. It's some next level inception bs. All I know is that Python seems to get increasingly slower the more I use threading.

You won't get a performance boost with threading alone. You need to either use the multiprocessing module or you need to unlock the GIL (using numba or Cython or something else)

If you're interested here's a good post talking about the GIL and effective use of threading in Python:
http://python-notes.curiousefficiency.org/en/latest/python3/multicore_python.html

# ? Oct 19, 2016 11:49

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

You seem to be swapping terminology a bit.

A green thread isn't the same thing as a thread.

Threads are provided by the OS, green threads are provided by you (well usually by a library you're using like eventlet/gevent/asyncio) and all of them run in a single traditional thread.

I haven't really thought about it, but I can't think of any reason the GIL will have any effect on green threads since they all run in one regular thread.

Here's a quickly googled and skimmed explanation of different ways of doing concurrency in Python: http://stupidpythonideas.blogspot.com/2015/01/greenlets-threads-and-processes.html

(I think I got that all right, haven't done anything requiring threading in a couple years. I forget everything if I haven't used it in 6 months or more)

Thermopyle fucked around with this message at 19:12 on Oct 19, 2016

# ? Oct 19, 2016 19:08

Eela6: May 25, 2007; Shredded Hen

If you're looking for more information on Python re: concurrency, threads, async, and parallelism, the Python Cookbook (3rd ed), Fluent Python, and Essential Python all cover it pretty well.

# ? Oct 19, 2016 20:12

Jose Cuervo: Aug 25, 2004

I have two machines, and for each machine I have the intervals during which it was working as a list of the start points and list of the end points of the intervals (for example [0, 5, 9] and [3, 6, 13] would indicate that the machine was working from time 0 to 3, 5 to 6, and 9 to 13).

I would like to determine how to calculate the start and end points of intervals where exactly one machine was working and exactly two machines were working are the same time.

For instance if the start and end points of the second machine intervals were [0, 7] and [2, 10] then the start and end points of intervals where exactly one machine was working are [2, 5, 7, 10] and [3, 6, 9, 13] and the start and end points of intervals where exactly two machines were working are [0, 9] and [2, 10].

I would like to be able to generalize this to three or more machines as well.

Any pointers on how to get started?

# ? Oct 21, 2016 20:20

SurgicalOntologist: Jun 17, 2004

I haven't had an opportunity to use it but I came across this library recently that appears to be perfect for your problem.

# ? Oct 21, 2016 20:54

mystes: May 31, 2006

Jose Cuervo posted:

I have two machines, and for each machine I have the intervals during which it was working as a list of the start points and list of the end points of the intervals (for example [0, 5, 9] and [3, 6, 13] would indicate that the machine was working from time 0 to 3, 5 to 6, and 9 to 13).

I would like to determine how to calculate the start and end points of intervals where exactly one machine was working and exactly two machines were working are the same time.

For instance if the start and end points of the second machine intervals were [0, 7] and [2, 10] then the start and end points of intervals where exactly one machine was working are [2, 5, 7, 10] and [3, 6, 9, 13] and the start and end points of intervals where exactly two machines were working are [0, 9] and [2, 10].

I would like to be able to generalize this to three or more machines as well.

Any pointers on how to get started?

Just store a position in each list and state for each computer and for each iterations take the smallest value from the values at the current positions and toggle the state of the computer that list belongs to and increment the position for that list. (If multiple lists have that value at the current position then toggle/increment all of them.)The value you took is then the current time and you have the state of all the computers at that time. Repeat until you have reached the end of all the lists.

mystes fucked around with this message at 21:38 on Oct 21, 2016

# ? Oct 21, 2016 21:36

Jose Cuervo: Aug 25, 2004

mystes posted:

Just store a position in each list and state for each computer and for each iterations take the smallest value from the values at the current positions and toggle the state of the computer that list belongs to and increment the position for that list. (If multiple lists have that value at the current position then toggle/increment all of them.)The value you took is then the current time and you have the state of all the computers at that time. Repeat until you have reached the end of all the lists.

I think I understand what you described, and implemented it as follows:

Python code:

overlapping_interval_starts = {}
overlapping_interval_ends = {}
whole_list = [(start, 's') for start in interval_starts] + [(end, 'e') for end in interval_ends]
whole_list.sort()

count = 1
if whole_list:
    previous, _ = whole_list.pop(0)
    for elt, elt_type in whole_list:
        current = elt
        if (count != 0) and (previous != current):
            try:
                overlapping_interval_starts[count].append(previous)
                overlapping_interval_ends[count].append(current)
            except KeyError:
                overlapping_interval_starts[count] = [previous]
                overlapping_interval_ends[count] = [current]

        if elt_type == 's':
            count += 1
        else:
            count -= 1
        previous = current

It works just fine on the 2 machine problem I posted but on this problem:

Python code:

Machine 1: [2, 7], [6, 11]
Machine 2: [1, 5, 11], [3, 10, 13]
Machine 3: [2, 6], [5, 8]

yields:

Python code:

overlapping_interval_starts = {1: [1, 10, 11],
                               2: [3, 5, 6, 8],
                               3: [2, 7]}
overlapping_interval_ends = {1: [2, 11, 13],
                             2: [5, 6, 7, 10],
                             3: [3, 8]}

which is correct but I would really like for there to be two intervals (3 to 7 and 8 to 10) where two machines overlap. Any thoughts on how I can accomplish this, and whether my code looks correct as far as the original problem?

# ? Oct 22, 2016 02:20

SurgicalOntologist: Jun 17, 2004

Seriously check out traces. Your problem is literally their first example on the first page of their documentation.

# ? Oct 22, 2016 04:11

mystes: May 31, 2006

Oh I misread your earlier post to be saying you had lists like [start1, end1, start2, end2] rather than like [[start1, start2], [end1, end2]]. Anyway, using traces as suggested by SurgicalOntologist:

code:

import traces

def tuple_to_ts((starts, ends)):
	ts = traces.TimeSeries()
	for t in starts:
		ts[t] = 1
	for t in ends:
		ts[t] = 0
	return ts

def find_two_machine_intervals(tuples):
	trace_list = map(tuple_to_ts, tuples)
	count = traces.TimeSeries.merge(trace_list, operation=sum)
	return [ (start, end) for ((start, startcount), (end, endcount)) in count.iterintervals() if startcount == 2]

code:

>>> tuples = [
([2, 7], [6, 11]),
([1, 5, 11], [3, 10, 13]),
([2, 6], [5, 8])
]

>>> find_two_machine_intervals(tuples)
[(3, 7), (8, 10)]

mystes fucked around with this message at 15:13 on Oct 22, 2016

# ? Oct 22, 2016 12:45

Master_Odin: Apr 15, 2010; My spear never misses its mark...

ladies

Does anyone have experience in hosting their own private PyPI setup? Ideally, this would be something I could run completely via a python script that doesn't require some apache/nginx configuration as I'm not sure I have access to that on the cloud provider I'm hoping to use for this.

# ? Oct 22, 2016 17:02

sarcastx: Feb 26, 2005

I have a quick question involving Halloween, a Raspberry Pi, and a doorbell.

I'm building a Halloween "prank" doorbell that involves a Raspberry Pi with a servo controller, some relays and a pushbutton - essentially what happens is that my plain old doorbell is replaced with a doorbell that activates a Raspberry Pi, which performs one of five spoopy activities randomly when people come to trick or treat. These spoops invove sounds, lights and servos. The whole show is run in Python, which I'm very new to.

What I'm having an issue with is the behavior of "while True :" loops, and the use of functions/GPIO calls - so I wasn't sure whether to post here or in the Raspberry Pi thread but I thought I'd start here.

Further up in the code, I've built my functions; the five "doorbell_#" functions (where # is 1-5), the "ringring" function which selects one of the five randomly, and another function which pulses the LED on the doorbell itself (this is to mimic a heartbeat or breathing or something). It looks very cool when it works; the PWM controller board I have attached to the Pi gives it a very cool looking up & down ebb - plus I changed the LED in the doorbell itself from white to red.

The issue is that I can either:
1). get my doorbell "heartbeat" function to loop properly - but then the doorbell button only registers presses in between loops, or
2). get my doorbell "heartbeat" to play one time only, and the doorbell button works any time after the "heartbeat" finishes

It looks like the issue is that the "doorbellheartbeat()" function running is preventing the next line (waiting for the button to be pushed) from running. With the code as below (using RPi.GPIO) the heartbeat runs once and then the button works immediately afterwards (scenario #2 above):

code:

while True :
    input_state = GPIO.input(24)
    doorbellheartbeat()
    GPIO.wait_for_edge(24, GPIO.FALLING)
    ringring()
    time.sleep(5)

but with the code like this, the heart beats repeatedly and the button only works between loops (scenario #1 above):

code:

while True :
    input_state = GPIO.input(24)
    if input_state == False:
        ringring()
        time.sleep(5)
    else :
        doorbellheartbeat()

any ideas how I can have my cake and eat it too?

# ? Oct 22, 2016 17:07

SurgicalOntologist: Jun 17, 2004

It's been awhile since I used GPIO but I think you want to look into GPIO.add_event_detect.

wait_for_edge blocks, which prevents the heartbeat from running. Your second attempt only checks the state at one specific time. Whereas add_event_detech basically gives you a flag as to whether an event occurred any time since you last checked.

Edit: The more robust option is to do everything in asyncio but it looks like add_event_detect gives you some basic asynchronous functionality which is all you need.

SurgicalOntologist fucked around with this message at 17:23 on Oct 22, 2016

# ? Oct 22, 2016 17:19

huhu: Feb 24, 2006

Two things I'd suggest looking into are for loops which will run a loop a set amount of times and interupts which can stop other functions. I'm new to gpio stuff so you'll have to do some searching.

# ? Oct 22, 2016 17:21

huhu: Feb 24, 2006

Oops

# ? Oct 22, 2016 17:36

Eela6: May 25, 2007; Shredded Hen

with regards to jose quervo's question:

This is how I would do it. Included is a test for your example. This is pure python - not familiar with the library mentioned. (Please indulge my love for namedtuples.)

Python code:

def count_machine_intervals(machines):
    START = 'START'
    END = 'END'
    Event = namedtuple('Event', ['time', 'type'])
    Interval = namedtuple('Interval', ['start', 'end'])
    def _generate_events():
        for machine in machines:
            for startTime in machine[0]:
                yield Event(startTime, START)
            for endTime in machine[1]:
                yield Event(endTime, END)

    events = sorted(_generate_events(), key = lambda x: x.time)
    count = 0
    previousTime = 0
    machineCountIntervals = defaultdict(list)
    def _update_machine_count_intervals():

        if interval.start == interval.end:
            return
        if machineCountIntervals[count]:
            # try merging intervals if appropriate
            lastSeen = machineCountIntervals[count][-1]
            if lastSeen.end == interval.start:
                machineCountIntervals[count][-1] = Interval(lastSeen.start, interval.end)
                return
        machineCountIntervals[count].append(interval)
        
    for event in events:
        interval = Interval(previousTime, event.time)
        _update_machine_count_intervals()

        if event.type == START:
            count += 1
        else:
            count -= 1
        previousTime = interval.end
    return machineCountIntervals

def test():
    machine_00 = [2, 7], [6, 11]
    machine_01 = [1, 5, 11], [3, 10, 13]
    machine_02 = [2, 6], [5, 8]
    machines = (machine_00, machine_01, machine_02)
    count = count_machine_intervals(machines)
    for key in count:
        print(key, count[key])

test()

output:

code:

0 [Interval(start=0, end=1)]
1 [Interval(start=1, end=2), Interval(start=10, end=13)]
2 [Interval(start=3, end=7), Interval(start=8, end=10)]
3 [Interval(start=2, end=3), Interval(start=7, end=8)]

edit: some small formatting changes.

Eela6 fucked around with this message at 20:46 on Oct 22, 2016

# ? Oct 22, 2016 20:30

baka kaba: Jul 19, 2003; PLEASE ASK ME, THE SELF-PROFESSED NO #1 PAUL CATTERMOLE FAN IN THE SOMETHING AWFUL S-CLUB 7 MEGATHREAD, TO NAME A SINGLE SONG BY HIS EXCELLENT NU-METAL SIDE PROJECT, SKUA, AND IF I CAN'T PLEASE TELL ME TO
EAT SHIT

I've never used Pandas, but I'm giving it a go (mainly to spit out some charts) and I'm wondering what the general approach is for this problem

I have some JSON data that's a mix of values and dicts, and I'm using pd.read_json to turn it into a dataframe. I want to filter rows based on a value in the dicts, but it looks like df[df.thing['thing_id']] won't work, because it's using the column data as hashed keys and you can't do that with a dict? I could filter the incoming rows by hand before creating a dataframe, but that seems a bit silly since pandas is entirely about working with datasets to get exactly what you want.

And I could manually expand all the dicts into individual columns, but that seems a bit awkward too. What's the general approach to filtering this kind of thing?

I'd post some of the JSON but the API site is now down :thumbsup:

But hopefully you get what I mean

# ? Oct 22, 2016 23:02

SurgicalOntologist: Jun 17, 2004

The correct way to do what you're trying to do is df[df.thing == 'thing_id']. Or do you have dictionaries in each cell? If so you need to flatten that out before constructing the dataframe.

# ? Oct 22, 2016 23:16

baka kaba: Jul 19, 2003; PLEASE ASK ME, THE SELF-PROFESSED NO #1 PAUL CATTERMOLE FAN IN THE SOMETHING AWFUL S-CLUB 7 MEGATHREAD, TO NAME A SINGLE SONG BY HIS EXCELLENT NU-METAL SIDE PROJECT, SKUA, AND IF I CAN'T PLEASE TELL ME TO
EAT SHIT

(Sorry, I meant df[df.thing['thing_id'] == 123], filtering on rows that contain a dict with a specific value on a key)

Yeah each JSON object is like

JSON code:

"3377":{
   "id":3377,
   "date":{
      "date":"2016-10-17 00:00:00.000000",
      "timezone_type":3,
      "timezone":"Europe\/London"
   },
   "company":{
      "code":"IPSOS",
      "name":"Ipsos MORI",
      "canonical":"ipsos-mori"
   },
   "client":"Evening Standard",
   "type":{
      "id":1,
      "code":"WESTVI",
      "name":"Westminster Voting Intention",
      "country":"GBR",
      "canonical":"westminster",
      "menu":"Westminster",
      "short_name":"Westminster VI",
      "category":1,
      "hashtag":"#GE2020",
      "partylist":[
         "CON",
         "LAB",
         "LD",
         "UKIP",
         "GRN"
      ],
      "complist":[

      ]
   },
   "update_time":{
      "date":"2016-10-19 15:26:46.000000",
      "timezone_type":3,
      "timezone":"Europe\/London"
   },
   "tablesurl":"https:\/\/www.ipsos-mori.com\/Assets\/Docs\/Polls\/pm-october-2016-tables.pdf",
   "mode":null,
   "headline":{
      "CON":{
         "pct":47,
         "party":"CON",
         "code":"CON"
      },
      "GRN":{
         "pct":4,
         "party":"GRN",
         "code":"GRN"
      },
      "LAB":{
         "pct":29,
         "party":"LAB",
         "code":"LAB"
      },
      "LD":{
         "pct":7,
         "party":"LD",
         "code":"LD"
      },
      "UKIP":{
         "pct":6,
         "party":"UKIP",
         "code":"UKIP"
      }
   },
   "previous_poll":null
}

so there's a lot of nested data in there, so some columns end up containing dictionaries. So it's really meant to be flattened when you do the read_json command? I know there are parameters for running formatting functions and the like

I mean it makes sense, I just want to do it the 'right' way - pandas looks cool, but most of it is way over my head, it feels like a lot of documentation takes your understanding of data analysis for granted

# ? Oct 22, 2016 23:43

SurgicalOntologist: Jun 17, 2004

Yes, pandas is going to have a hard time with such a complex structure. In particular it can't handle dictionaries in each cell very well. I mean you could do something like

Python code:

from operator import itemgetter

df[df.thing.apply(itemgetter('thing_id')) == 123]

But you're going to have a much nicer time if you flatten. Try this: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.io.json.json_normalize.html
and if that can't handle JSON in your specific structure you'll have to write a custom flattener. Shouldn't be too difficult and will give you the most control.

# ? Oct 23, 2016 00:33

QuarkJets: Sep 8, 2008

sarcastx posted:

I have a quick question involving Halloween, a Raspberry Pi, and a doorbell.

I'm building a Halloween "prank" doorbell that involves a Raspberry Pi with a servo controller, some relays and a pushbutton - essentially what happens is that my plain old doorbell is replaced with a doorbell that activates a Raspberry Pi, which performs one of five spoopy activities randomly when people come to trick or treat. These spoops invove sounds, lights and servos. The whole show is run in Python, which I'm very new to.

What I'm having an issue with is the behavior of "while True :" loops, and the use of functions/GPIO calls - so I wasn't sure whether to post here or in the Raspberry Pi thread but I thought I'd start here.

Further up in the code, I've built my functions; the five "doorbell_#" functions (where # is 1-5), the "ringring" function which selects one of the five randomly, and another function which pulses the LED on the doorbell itself (this is to mimic a heartbeat or breathing or something). It looks very cool when it works; the PWM controller board I have attached to the Pi gives it a very cool looking up & down ebb - plus I changed the LED in the doorbell itself from white to red.

The issue is that I can either:
1). get my doorbell "heartbeat" function to loop properly - but then the doorbell button only registers presses in between loops, or
2). get my doorbell "heartbeat" to play one time only, and the doorbell button works any time after the "heartbeat" finishes

It looks like the issue is that the "doorbellheartbeat()" function running is preventing the next line (waiting for the button to be pushed) from running. With the code as below (using RPi.GPIO) the heartbeat runs once and then the button works immediately afterwards (scenario #2 above):
code:
while True :
    input_state = GPIO.input(24)
    doorbellheartbeat()
    GPIO.wait_for_edge(24, GPIO.FALLING)
    ringring()
    time.sleep(5)
but with the code like this, the heart beats repeatedly and the button only works between loops (scenario #1 above):
code:
while True :
    input_state = GPIO.input(24)
    if input_state == False:
        ringring()
        time.sleep(5)
    else :
        doorbellheartbeat()
any ideas how I can have my cake and eat it too?

It sounds like what you want is to listen for doorbell inputs while running the doorbell heartbeat simultaneously. One way to accomplish this is with threads:

Python code:

from time import sleep
from threading import Thread

def door_bell_heartbeat():
    while True:
        # Do the doorbell heartbeat thing
        print('The shriek, I said, was my own in a dream')
        sleep(2)

def door_bell_ringing():
    while True:
        # Check doorbell input state
        print('DING DONG')
        sleep(3)

def main():
    t_doorbell = Thread(target=door_bell_ringing)
    t_heartbeat = Thread(target=door_bell_heartbeat)
    t_doorbell.start()
    t_heartbeat.start()

# ? Oct 23, 2016 02:08

sarcastx: Feb 26, 2005

QuarkJets posted:

Python code:
thread stuff

That did it! Thanks QuarkJets!

# ? Oct 23, 2016 15:00

baka kaba: Jul 19, 2003; PLEASE ASK ME, THE SELF-PROFESSED NO #1 PAUL CATTERMOLE FAN IN THE SOMETHING AWFUL S-CLUB 7 MEGATHREAD, TO NAME A SINGLE SONG BY HIS EXCELLENT NU-METAL SIDE PROJECT, SKUA, AND IF I CAN'T PLEASE TELL ME TO
EAT SHIT

SurgicalOntologist posted:

But you're going to have a much nicer time if you flatten. Try this: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.io.json.json_normalize.html
and if that can't handle JSON in your specific structure you'll have to write a custom flattener. Shouldn't be too difficult and will give you the most control.

Yeah this looks workable once I get the hang of it, thanks!

# ? Oct 23, 2016 19:22

QuarkJets: Sep 8, 2008

sarcastx posted:

That did it! Thanks QuarkJets!

Hey no problem

I wrote that up after a few beers and looking back at my implementation, you really only need to create one thread; you could stick the heartbeat calls in a spawned thread and keep the doorball listening in a while loop in main(). In other words, the blurb I wrote uses 3 threads (the main thread + 2 spawned threads) but you really only need 2. Although there's no harm in having 3!

# ? Oct 23, 2016 20:18

Jose Cuervo: Aug 25, 2004

SurgicalOntologist posted:

Seriously check out traces. Your problem is literally their first example on the first page of their documentation.

I looked at it but was having a hard time figuring out how to get started with it. Luckily mystes post showed me that things were much simpler than I thought.

mystes posted:

Oh I misread your earlier post to be saying you had lists like [start1, end1, start2, end2] rather than like [[start1, start2], [end1, end2]]. Anyway, using traces as suggested by SurgicalOntologist:
code:
code goes here...

Thanks! This works with the exception that I had to add the default=0 parameter when generating each traces.TimeSeries to get it to correctly compute the intervals where only one machine was working (because otherwise it assumes the default value for times prior to the first time stamp are equal to the value at the time stamp):

Python code:

import traces

def tuple_to_ts((starts, ends)):
	ts = traces.TimeSeries(default=0)
	for t in starts:
		ts[t] = 1
	for t in ends:
		ts[t] = 0
	return ts

Eela6 posted:

with regards to jose quervo's question:

This is how I would do it. Included is a test for your example. This is pure python - not familiar with the library mentioned. (Please indulge my love for namedtuples.)
Python code:
code goes here
edit: some small formatting changes.

Thanks for taking the time to write this! I am going to go with the traces library solution as it seems like less code for me to write/deal with.

# ? Oct 24, 2016 18:00

Eela6: May 25, 2007; Shredded Hen

Makes sense! A well-regarded library is always a good way to go. Besides being a little bulky, there is probably some corner case I've missed in my implementation. That said, I enjoyed writing it and figuring out the 'tricks ' (like ignoring zero-width intervals and merging intervals of the form n: (a, b) , n:(b, c) --> n:(a, c)

# ? Oct 24, 2016 18:15

cinci zoo sniper: Mar 15, 2013

Is there a laconic way to return every X out of every Y elements of the list? Say, with X=2 and Y=4, output of foo([1, 2, 3, 4, 5, 6], X, Y) would be [1, 2, 5, 6]. I can write a for loop doing but, but it's going to be way too slow.

# ? Oct 27, 2016 13:53

Tigren: Oct 3, 2003

cinci zoo sniper posted:

Is there a laconic way to return every X out of every Y elements of the list? Say, with X=2 and Y=4, output of foo([1, 2, 3, 4, 5, 6], X, Y) would be [1, 2, 5, 6]. I can write a for loop doing but, but it's going to be way too slow.

If I understand your question, you're looking for the basic slicing feature of python lists.

Python code:

In [1]: foo = [1,2,3,4,5,6,7,8,9,10]

In [2]: foo
Out[2]: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

In [3]: foo[::2]
Out[3]: [1, 3, 5, 7, 9]

In [4]: foo[1::2]
Out[4]: [2, 4, 6, 8, 10]

In [5]: foo[::-1]
Out[5]: [10, 9, 8, 7, 6, 5, 4, 3, 2, 1]

Tigren fucked around with this message at 15:10 on Oct 27, 2016

# ? Oct 27, 2016 15:07

overeager overeater: Oct 16, 2011; "The cosmonauts were transfixed with wonderment as the sun set - over the Earth - there lucklessly, untethered Comrade Todd on fire."

cinci zoo sniper posted:

Is there a laconic way to return every X out of every Y elements of the list? Say, with X=2 and Y=4, output of foo([1, 2, 3, 4, 5, 6], X, Y) would be [1, 2, 5, 6]. I can write a for loop doing but, but it's going to be way too slow.

code:

def XofEveryY(iter, x, y):
    return [j for (i, j) in enumerate(iter) if (i%y) < x]

# ? Oct 27, 2016 15:08

cinci zoo sniper: Mar 15, 2013

Tigren posted:

If I understand your question, you're looking for the basic slicing feature of python lists.
Python code:
In [1]: foo = [1,2,3,4,5,6,7,8,9,10]

In [2]: foo
Out[2]: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

In [3]: foo[::2]
Out[3]: [1, 3, 5, 7, 9]

In [4]: foo[1::2]
Out[4]: [2, 4, 6, 8, 10]

In [5]: foo[::-1]
Out[5]: [10, 9, 8, 7, 6, 5, 4, 3, 2, 1]

I'm already using slicing, the task there is to expand slicing so it operates with groups of multiple points, which is not achievable with generic slicing in Python. As you can see in your own examples, you first get every second element, then every second element starting with the second, whereas the question was to get every few from every several samples, which overeager overeater's answer addresses.

overeager overeater posted:

code:

def XofEveryY(iter, x, y):
    return [j for (i, j) in enumerate(iter) if (i%y) < x]

Thank you!

# ? Oct 27, 2016 15:46

Eela6: May 25, 2007; Shredded Hen

Overager's solution is good but creates a new list. You can use a generator comprehension to lazily yield new entries, which avoids a lot of extra overhead from creating an intermediate list. If you do need an explicit list rather than iterator, you can just call list() on the generator, or use a list comprehension later.

This is accomplished by replacing the brackets [] with parentheses ()

Python code:

def gen_x_of_every_y(iter, x, y):
    return (j for (i, j) in enumerate(iter) if (i%y) < x)

# ? Oct 27, 2016 20:28

cinci zoo sniper: Mar 15, 2013

Eela6 posted:

Overager's solution is good but creates a new list. You can use a generator comprehension to lazily yield new entries, which avoids a lot of extra overhead from creating an intermediate list. If you do need an explicit list rather than iterator, you can just call list() on the generator, or use a list comprehension later.

This is accomplished by replacing the brackets [] with parentheses ()
Python code:
def gen_x_of_every_y(iter, x, y):
    return (j for (i, j) in enumerate(iter) if (i%y) < x)

Yeah this'll help, dataset that goes into iterable there can be rather large.

# ? Oct 27, 2016 20:35

LochNessMonster: Feb 3, 2005; I need about three fitty

I'm trying to generate a page for each dealer in my sqlite db, which kinda works, but when trying to display all vehicles for that dealer I'm running into some issues.

I think my 2nd sql query isn't picking up the dealer name. And if it would, I'm not sure if it'd actually return all vehicles for each page. I'm kinda stuck on how to figure this out.

Python code:

@app.route('/dealers/<dealerName>/')
def show_dealer(dealerName):
    dealerName = query_db('select distinct dealer from motorcycles')
    for dealer in dealerName:
        dealerVehicles = query_db("select * from motorcycles where dealer = %s") % (dealer)
    return render_template('layout.html', dealerName=dealerName, dealerVehicles=dealerVehicles)

# ? Nov 2, 2016 20:21

Adbot: ADBOT LOVES YOU

# ? May 8, 2024 10:07

Tigren: Oct 3, 2003

LochNessMonster posted:

I'm trying to generate a page for each dealer in my sqlite db, which kinda works, but when trying to display all vehicles for that dealer I'm running into some issues.

I think my 2nd sql query isn't picking up the dealer name. And if it would, I'm not sure if it'd actually return all vehicles for each page. I'm kinda stuck on how to figure this out.
Python code:
@app.route('/dealers/<dealerName>/')
def show_dealer(dealerName):
    dealerName = query_db('select distinct dealer from motorcycles')
    for dealer in dealerName:
        dealerVehicles = query_db("select * from motorcycles where dealer = %s") % (dealer)
    return render_template('layout.html', dealerName=dealerName, dealerVehicles=dealerVehicles)

Your string substitution is out of place.

dealerVehicles = query_db("select * from motorcycles where dealer = %s" % (dealer))

If you're using python3, take a look at .format() instead. https://pyformat.info/

Tigren fucked around with this message at 21:17 on Nov 2, 2016

# ? Nov 2, 2016 21:11

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »