Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
breaks
May 12, 2001

Sad Panda posted:

Three examples of sites that refuse to work (claiming not visible/cannot focus) without some form of skullduggery include

Sorry for the late reply. I was under the weather for a couple days then had a lot of poo poo goin on what with the holiday.


Yeah, this is dumb. It just needs a click in the right spot to make the real input visible. If you really wanted to avoid the javascript you could click its parent element "div.username_wrap div.textboxex_Center". In some similar cases you might need ActionChains for more fine grained control over the click. But for your usage I think your js approach is actually the right thing to do. You're not trying to validate that this dual input UI implementation works normally, and it's surely less fragile than precise clicking.

quote:

2) https://www.kingjackcasino.com

The same issue. I can click on login, but then can't interact with the username field. I've tried
HTML code:
[class="login-box__input-wrapper"]
and the Chrome generated selector which is...
HTML code:
#ui-login-form > div:nth-child(1) > input[type="text"]

Actually a different issue. There are multiple elements matching both of those selectors and the first one it's finding in the DOM is not the one you want. "#login-modal input[placeholder="User Name / Email"]" is a pretty dumb selector but it gets the right one and I don't see a less fragile option at first glance.

quote:

3) Nationwide Online Banking. (https://onlinebanking.nationwide.co.uk/AccessManagement/Login). If you type in a random 10 digit customer number, click on Log in using memorable data, type a random thing into memorable data then the passnumber dropdowns become interactable. However, the dropdown values are hidden with aria-hidden = true
JavaScript code:
"document.getElementsByClassName('selection-list__value')[0].setAttribute('aria-hidden', 'false');"
That allows me to interact with them using Selenium.

Here the right thing to do is use selenium.webdriver.support.select.Select with the actual select element. Along the lines of this:

code:
from selenium.webdriver.support.select import Select
first_digit = Select(driver.find_element_by_css_selector('[name=FirstPassnumberValue]'))
first_digit.select_by_value('3')

breaks fucked around with this message at 06:09 on May 30, 2018

Adbot
ADBOT LOVES YOU

Sad Panda
Sep 22, 2004

I'm a Sad Panda.
Thank you so much, those are super detailed answers and a great insight into using Selenium.

My latest joy has been trying to use ActionChains to click buttons in a HTML5 game (https://www.netent.com/en/game/starburst-2/). I wrote the following function.

Python code:
def click_location(driver, x, y):
    print("click_location")
    action = ActionChains(driver)
    action.move_by_offset(x/2, y/2)
    print(f"About to click {x}, {y}.")
    action.click()
    action.perform()
Note the halving x and y is because of using this on a Macbook Retina which makes the co-ordinates go funny. I tried this across 3 browsers.
1. Safari. Perfect. Clicked as hoped and could be looped as many times to click in the same place.
2. Firefox. Nope. move_by_offset (which under Safari seems to basically be move_to_coords) seems to work as 'move from last place the cursor was set' so while it puts it in the right place initially, the 2nd time it moves it out of range and throws an error.
3. Chrome - Does nothing. No clicking. No errors. No sign of why it's being like that.

I came up with a pyautogui alternative, but that Chrome experience was infuriating.

breaks
May 12, 2001

Yeah cross browser compatibility is better than it was a year ago but still has some issues. The transition to the w3c webdriver standard plus the slow pace of development on this stuff kind of threw a wrench in.

On my phone so I didn’t look at the page, but you might try move_to_element_with_offset using whatever container element for the game or even body, instead of move_by_offset.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
Is there a way to run PEP8 enforcement/linting only on code changed in a commit? We're a little scared about turning that stuff on and having to reformatting a bunch of files to do possibly ten lines of changes.

huhu
Feb 24, 2006
Anyone else observe PyCharm coming to a standstill for even the most basic of operations? I started a new project and was working in Sublime. I opened the same directory in Pycharm and it's taking me like ~1-2 seconds to open a directory to see what's inside. I don't have any other processes running within it and the Activity Monitor for mac is reporting up to 300% CPU usage.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!

huhu posted:

Anyone else observe PyCharm coming to a standstill for even the most basic of operations? I started a new project and was working in Sublime. I opened the same directory in Pycharm and it's taking me like ~1-2 seconds to open a directory to see what's inside. I don't have any other processes running within it and the Activity Monitor for mac is reporting up to 300% CPU usage.

Yeah, but it most often happens while indexing all the files in the directory or project. Did it cool off after awhile?

cinci zoo sniper
Mar 15, 2013




huhu posted:

Anyone else observe PyCharm coming to a standstill for even the most basic of operations? I started a new project and was working in Sublime. I opened the same directory in Pycharm and it's taking me like ~1-2 seconds to open a directory to see what's inside. I don't have any other processes running within it and the Activity Monitor for mac is reporting up to 300% CPU usage.

Initial indexing, yes. Check the running tasks in status bar.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

I only notice any sluggishness in PyCharm in the first 5-30 seconds while indexing.

Otherwise I can barely tell a difference between it and something quick like Notepad++.

Granted, I've got a decent system (i5-4670k with 32GB ram and a fast SSD).


Anyway, speaking of PyCharm, I was reporting a bug on their issue tracker today and noticed the filter for "Reported by me". Turns out that, since 2013, I've opened 173 issues there. A good portion of them have been fixed or implemented, a decent portion of them I was pointed to an alternate way of doing what I wanted, and vast majority of the rest I at least get a comment or some sort of activity from a developer...which is nice compared to a lot of bug trackers where it feels like you're yelling into the void.

Jetbrains is a good organization.

BaronVonVaderham
Jul 31, 2011

All hail the queen!

Thermopyle posted:

I only notice any sluggishness in PyCharm in the first 5-30 seconds while indexing.

Otherwise I can barely tell a difference between it and something quick like Notepad++.

Granted, I've got a decent system (i5-4670k with 32GB ram and a fast SSD).

:same: just with an AMD FX-9590 Black Edition.

titaniumone
Jun 10, 2001

I've got a style question for API design.

I'm working with pyVmomi, and the vmware web services API is pretty difficult to work with correctly until you know it very well. It's very misused in multiple projects in my organization and my team is about to start writing a vSphere-focused application, so before we start I'm writing a wrapper around pyVmomi. My goal is to encapsulate it for non-vmware developers and follow the principle of least surprise, e.g. VirtualMachine.powerOff() will block until the VM has powered off successfully. (as opposed to the pyVmomi API, which returns a Task object, which you need to write a lot of scaffolding to query correctly using a PropertyCollector unless you want to poll it in a busy-loop, and even that is easy to get wrong). This won't be a drop-in replacement for pyVmomi but it will be an alternative that is easier to use correctly.

I'm going to have some high method-count classes (like VirtualMachine) and I'm trying to think of ways to make the API less overwhelming.

I was thinking I could have dummy classes defined inside the VirtualMachine class which break it up into related groups (power, management, storage, cloning, etc) but then I have a bit of gross "internals sharing" to do, passing self around to these pseudo-child clases.

Here's an example. Is this worth bothering with, or should I just have a shitload of methods in my top level class? Or is there a nicer alternative I'm not seeing?
code:
class VirtualMachine:

	class PowerOps:
		def __init__(self, p):
			self._parent = p
			
		def on(self):
			task = self._parent._mor.PowerOnVM_Task()
			self._parent._service.wait_task_complete(task)
			
		# ... many more methods
	
	class ManagementOps:
		def __init__(self, p):
			self._parent = p
			
		def rename(self, new_name):
			task = self._parent._mor.Rename_Task(new_name)
			self._parent._service.wait_task_complete(task)
			
		# ... many more methods
		
	# ... many more class <thing>Ops
	
	def __init__(self, mor):
	
		self._mor = mor # Managed Object Reference, the vmware API object we're encapsulating
		self._service = get_service_singleton() # utilities for querying API nicely
	
		self.power = VirtualMachine.PowerOps(self)
		self.management = VirtualMachine.ManagementOps(self)

		# ...
		
		
# Usage:
vm = get_service_singleton().find_virtual_machine(name_contains="fart")
vm.management.rename("butt")
vm.power.on()

titaniumone fucked around with this message at 03:18 on Jun 1, 2018

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

I think the general idea is fine, but I wouldn't define the helper classes inside your VM class.

I think packages and modules are a better level for organization of different classes. You could have a vm.py with your VM class and then a power.py for your Power class, etc. all inside a vm package. Or just keep them in the same module as your VM class...kind of just depends on how many of these additional classes you have.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
Does anybody have recommendations for real-time viewing of Python logs? Cutelog might be good enough, but I figured there were a few Django people that have delved into this here and have something to say.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Rocko Bonaparte posted:

Does anybody have recommendations for real-time viewing of Python logs? Cutelog might be good enough, but I figured there were a few Django people that have delved into this here and have something to say.

What do you need exactly that tail -f doesn't do?

Typically Django people (if they're not just doing tail -f) will be dumping their logs to ELK or a hosted solution like papertrail.

The Fool
Oct 16, 2003


I love me some elk.



Oh wait, wrong forum.


ELK is good too, if you want a super low effort self hosted ELK style setup and have any docker experience, it is super easy to get Graylog setup in containers.

cinci zoo sniper
Mar 15, 2013




Thermopyle posted:

I only notice any sluggishness in PyCharm in the first 5-30 seconds while indexing.

Otherwise I can barely tell a difference between it and something quick like Notepad++.

Granted, I've got a decent system (i5-4670k with 32GB ram and a fast SSD).


Anyway, speaking of PyCharm, I was reporting a bug on their issue tracker today and noticed the filter for "Reported by me". Turns out that, since 2013, I've opened 173 issues there. A good portion of them have been fixed or implemented, a decent portion of them I was pointed to an alternate way of doing what I wanted, and vast majority of the rest I at least get a comment or some sort of activity from a developer...which is nice compared to a lot of bug trackers where it feels like you're yelling into the void.

Jetbrains is a good organization.

Yeah I don’t experience any performance issues outside indexing either.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!

Thermopyle posted:

What do you need exactly that tail -f doesn't do?

Typically Django people (if they're not just doing tail -f) will be dumping their logs to ELK or a hosted solution like papertrail.

I have something running jobs that was previously creating a new console window for each new module it started up. This drove me crazy but it was what previous users had expected. I would outright dismiss it, but I had done a thing where I'd print ASCII smiley or frowny faces in a console app each time something passed on another tool, and I found my end users kind of relied on seeing those on a Friday afternoon to figure out if their setup was stable enough for the weekend. So I want to set up something a little more powerful, more robust, and less likely to run away with a pile of shells taking over the screen.

bamhand
Apr 15, 2010
Is there something comparable to R Shiny in Python? My work is looking to create an interactive report generating/model building tool.

cinci zoo sniper
Mar 15, 2013




bamhand posted:

Is there something comparable to R Shiny in Python? My work is looking to create an interactive report generating/model building tool.

My work wanted something similar but we had competing proposals. Mine was to place a risky bet on https://plot.ly/products/dash/ (literally Shiny for Python), the competing one was Shiny.

Shiny proposal won over being a business tested technology, but it was proposed by a moron who busted all deadlines and got fired for it. I had the joy of double-timing a an alien to me Shiny code base into internal production.

Dash felt fairly cool, if a little bit awkward, back when I made demo dashboard on it.

I did also consider a few other Python-ish D3.js projects, including Bokeh (the one visible library in this "field", also developed by a goon), but ultimately they didn't seem to be as good to me for whatever reason.

cinci zoo sniper fucked around with this message at 17:14 on Jun 1, 2018

SurgicalOntologist
Jun 17, 2004

I've enjoyed developing Bokeh apps, and there's also HoloViews which (sort of) provides a higher-level interface over Bokeh.

I believe there's a smaller project that's more directly comparable to Shiny but it's escaping me atm.

And there's also the option of just using widgets in Jupyter Notebook--depending on the use case this may or may not be an option.

cinci zoo sniper
Mar 15, 2013




I don't remember what my issue was with Bokeh. Either some unsupported viz that was in business requirements that was suported elsewhere, or it had to ask me to do occasional JS. Or there were some egregious warcrimes in the API but I seldom take offence to that.

SurgicalOntologist
Jun 17, 2004

I mean, it's not perfect and it's still in active development, although I think 1.0 is coming soon. I've always been impressed with the improvements every time I come back to it, but it's not surprising one would choose another direction. And yes you may have had to some JS depending on your needs.

bamhand
Apr 15, 2010
I think our main concern with R is capacity. If we have 200gb of data we want to train a model on we don't want R to poo poo the bed.

Baronash
Feb 29, 2012

So what do you want to be called?
I am trying to write a program to help automate part of my job, and I'm not sure what the best way to do it is. Each week, I have to assign about 50 volunteers to different daytime programs and nighttime supervision responsibilities. I am essentially trying to give everyone their top choice of daytime program while minimizing situations where 2 people work together both during the day and at night. I base these assignments on each volunteer's ranking of daytime programs, as well as a variety of tweakable features (making sure someone is in a specific program, ensuring two particular people do/don't work together, etc).

The only solution I can think of is some sort of genetic algorithm with a pretty simple scoring function, but I feel like there has to be a simpler solution, or a library that might make this easier. Any suggestions?

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

You should ask that in the general programming thread. It will get more exposure and there's nothing python specific about it.

SurgicalOntologist
Jun 17, 2004

I think you can make that a linear programming problem. With Python the best option (last time I looked into it) was PuLP, which has a terrible interface but it does work. You could make a matrix of variables where each slot is one person in one program taking the value of 0 or 1. Add a constraint for each program (e.g. row in the matrix) that it sums to 2 (2 volunteers in each slot), that the columns (volunteers) sum to 1 for the day rows and 1 for the night rows. "Don't pair up the same volunteers twice" is going to be hard to formulate linearly but may be doable combinatorically (something like "the sum of these four cells must be < 4; the sum of these four cells must be < 4, etc.). That might be too many constraints... in which case maybe there's a more clever way to do it or you could make the pairing concept part of the objective function (and I think you'd have to look into a solver that can handle a nonlinear objective function.. but still something more direct than a GA should work).

When I solved a problem like this I put the LPVariable objects (representing the 0 or 1 values) in an actual numpy array and used dot products to build the constraints. Another approach would be to store the LPVariables in a dict with keys of (volunteer, program) tuples and build all the constraints by looping.

Proteus Jones
Feb 28, 2013



I’m not sure a 0 or 1 will work with the program selection, since it’s implied that it’s more or less a gradient on Most Preferred to Least Preferred. And the goal is to get everyone as close to Most Preferred as you can.

It will work on the pair/don’t pair, though.

E: Hmmm... or maybe settle the program assignations prior to the matrix creation?

Proteus Jones fucked around with this message at 03:11 on Jun 2, 2018

Methanar
Sep 26, 2013

by the sex ghost
Maybe assign each person an array of weights where the index of each element relates to one of the jobs and the value of the weight relates to a preference for the job.

Then run your favorite sorting algorithm to arrange everyone such that, overall, the greatest ratio of weight-preference being met is achieved.

Then post sorting, do a few overrides to make sure the pairing constraint is met.

SurgicalOntologist
Jun 17, 2004

Proteus Jones posted:

I’m not sure a 0 or 1 will work with the program selection, since it’s implied that it’s more or less a gradient on Most Preferred to Least Preferred. And the goal is to get everyone as close to Most Preferred as you can.
The 0 or 1 are the variables that the program is solving for; the objective function is the dot product of those variables and a vector of preference strength.

Edit: at least, that's the base of the objective function. Could add other considerations as long as they can be linearized (or quadtratic with the right solver, etc.).

SurgicalOntologist fucked around with this message at 03:47 on Jun 2, 2018

Proteus Jones
Feb 28, 2013



SurgicalOntologist posted:

The 0 or 1 are the variables that the program is solving for; the objective function is the dot product of those variables and a vector of preference strength.

Edit: at least, that's the base of the objective function. Could add other considerations as long as they can be linearized (or quadtratic with the right solver, etc.).

Yep. I completely misread.

:doh:

Butter Activities
May 4, 2018

Anyone know why this:

https://stackoverflow.com/questions/23287/largest-prime-factor-of-a-number/412942#412942

is better what seemed to me to be the obvious solution:

def factor(num):
->factors = [ ]
->for x in range(1,num):
->-> if num%x == 0:
->->-> factors.append(x)
->factors.append(num)
->return(factors)
factor(42)

Nippashish
Nov 2, 2005

Let me see you dance!

SpaceSDoorGunner posted:

Anyone know why this:

https://stackoverflow.com/questions/23287/largest-prime-factor-of-a-number/412942#412942

is better what seemed to me to be the obvious solution:

def factor(num):
->factors = [ ]
->for x in range(1,num):
->-> if num%x == 0:
->->-> factors.append(x)
->factors.append(num)
->return(factors)
factor(42)

Yours finds non-prime factors.

Master_Odin
Apr 15, 2010

My spear never misses its mark...

ladies

SpaceSDoorGunner posted:

Anyone know why this:

https://stackoverflow.com/questions/23287/largest-prime-factor-of-a-number/412942#412942

is better what seemed to me to be the obvious solution:

def factor(num):
->factors = [ ]
->for x in range(1,num):
->-> if num%x == 0:
->->-> factors.append(x)
->factors.append(num)
->return(factors)
factor(42)

Your algorithm returns something different? It returns all possible factors, not the prime factorization of the number.
code:
>>> def prime_factors(n):
...     """Returns all the prime factors of a positive integer"""
...     factors = []
...     d = 2
...     while n > 1:
...         while n % d == 0:
...             factors.append(d)
...             n /= d
...         d = d + 1
...     return factors
...
>>> def factor(num):
...     factors = []
...     for x in range(1, num):
...             if num % x == 0:
...                     factors.append(x)
...     factors.append(num)
...     return factors
...
>>> prime_factors(42)
[2, 3, 7]
>>> factor(42)
[1, 2, 3, 6, 7, 14, 21, 42]

Sad Panda
Sep 22, 2004

I'm a Sad Panda.
This is for a personal project, stored locally.

I'm trying to use a Fernet key to encrpyt some information. The idea is to store the key in my OS X Keychain (using keyring). I'm having a small issue. I can convert it to a string, put it in the Keyring, pull it back out of the keyring, but I can't then use it as a Fernet key because it throws "ValueError: Fernet key must be 32 url-safe base64-encoded bytes.".

Python code:
from cryptography.fernet import Fernet
import keyring
import base64

my_keyring = keyring.get_keyring()

def gen_key():
    '''Generates a Fernet Key and stores it in my Keyring.'''
    key = Fernet.generate_key()
    key = str(key)
    my_keyring.set_password('My Fernet', 'Banking Key', key)
    print(f"generate_key : {key}")
   #generate_key : b'T34wLOQoAL2CqOU9y3wRammz0U-yRSNJKk29UqEg7qc='

def get_key():
    ''' Pulls the key out and encodes it back to a byte from a string. '''
    key = my_keyring.get_password('My Fernet', 'Banking Key').encode()
    print(f"get_key : {key}")
    #get_key : b"b'T34wLOQoAL2CqOU9y3wRammz0U-yRSNJKk29UqEg7qc='"
    return key

def encrypt(key, to_encrypt):
    f = Fernet(key)
    # ValueError: Fernet key must be 32 url-safe base64-encoded bytes.
    token = f.encrypt(to_encrypt)
    return f, token

gen_key()
key = get_key()
fernet, my_token = encrypt(key, b'hello')

That's the current situation.
If instead of that, this works just fine if I manually set the key as a string. What's going wrong? I've spent hours banging my head against this.

Python code:
key = 'g0c-0M8lyOh1foUBWLqzbwIZ3G4Muo9r_BLJfquuW20='
fernet, my_token = encrypt(key, b'hello')
decrypt(fernet, my_token)
edit - Fixed it.
Python code:
    key = my_keyring.get_password('My Fernet', 'Banking Key')[1:-1]
Works.

Sad Panda fucked around with this message at 23:58 on Jun 2, 2018

Butter Activities
May 4, 2018

Nippashish posted:

Yours finds non-prime factors.

Oh I'm dumb thanks.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

I stumbled across this site which I thought people might find useful.

It's basically an in-depth guide or tutorial of what looks like the whole itertools module. So, if you're like me and ever see someone using itertools and think "drat, I should use itertools more", maybe this will help.

Lumpy
Apr 26, 2002

La! La! La! Laaaa!



College Slice
Question about code running as a uwsgi flask app vs. running directly, and why one times out and one does not.

I have a tensorflow model and want to classify images. I have two versions of a script on a server that contain 99% the same code, but one has the extra lines of code to make it a flask app.

If I run the non-flask version at the command line, it classifies an image (from disk) in less than a second. Hooray! When I hit the Flask version via browser, it hangs and eventually times out. I know the Flask setup is fine: if I replace my app code with a simple template render and hit it in the browser, that works. I have increased the timeout to 120 seconds, and it still hangs. What can I do to troubleshoot this? I am an idiot with this stuff, so I'm not sure what to ask exactly or what information would be helpful to get an answer.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Throw lots of logging statements in there and see if it's your code or whether it's a framework thing causing your trouble.

Lumpy
Apr 26, 2002

La! La! La! Laaaa!



College Slice

Thermopyle posted:

Throw lots of logging statements in there and see if it's your code or whether it's a framework thing causing your trouble.

It gets to the line where it classifies the image and just sits there for a while then times out. That same code runs in under a second on the same machine not under uwsgi. So to my mind, that means it is somehow running slow or "different" in that capacity. But how or why.... :iiam:

SurgicalOntologist
Jun 17, 2004

It could be that something you're doing is trying to use an extra worker, which is not provisioned.

I had a similar problem with a Flask app once, and it ended up being a utility that bakes CSS into email was trying to use a second process. It would hang with only one worker (the default for the flask dev server) but worked fine with 2 or more.

Adbot
ADBOT LOVES YOU

Hed
Mar 31, 2004

Fun Shoe

Thermopyle posted:

I stumbled across this site which I thought people might find useful.

It's basically an in-depth guide or tutorial of what looks like the whole itertools module. So, if you're like me and ever see someone using itertools and think "drat, I should use itertools more", maybe this will help.

Thanks, I was dorking around this weekend and used itertools and kind of thought the same "why don't I reach for this more often".

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply