Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Munkeymon
Aug 14, 2003

Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.



Hughlander posted:

https://msdn.microsoft.com/en-us/library/ff628587(v=vs.93).aspx Looks like .NET even has a constraints library that will let me set it up really easily.

Holy poo poo - it's not often I find out about some new corner of .Net and am shocked at how neat it is.

Adbot
ADBOT LOVES YOU

huhu
Feb 24, 2006
Adobe Lightroom has certain files that will corrupt if Google Drive is running while it's open. What would be the easiest way to either get a notification or have Google drive close if Lightroom opens?

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

huhu posted:

Adobe Lightroom has certain files that will corrupt if Google Drive is running while it's open. What would be the easiest way to either get a notification or have Google drive close if Lightroom opens?

Very easy to write a little script in AutoIT to do this if you're on Windows.

Maha
Dec 29, 2006
sapere aude
Do any regex wizards know how to convert a list that looks like this:
code:
		[507] = "Fenros",
		[3535] = "Blackmoss the Fetid",
		[22060] = "Fenissa the Assassin",
		[1260] = "Great Father Arctikus",
		[8503] = "Gibblewilt",
		[16180] = "Shadikith the Glider",
		[16184] = "Nerubian Overseer",
...into this?

code:
["FENROS"] = true,
["BLACKMOSS THE FETID"] = true,
["FENISSA THE ASSASSIN"] = true,
(and so on)

leper khan
Dec 28, 2010
Honest to god thinks Half Life 2 is a bad game. But at least he likes Monster Hunter.

Maha posted:

Do any regex wizards know how to convert a list that looks like this:
code:

		[507] = "Fenros",
		[3535] = "Blackmoss the Fetid",
		[22060] = "Fenissa the Assassin",
		[1260] = "Great Father Arctikus",
		[8503] = "Gibblewilt",
		[16180] = "Shadikith the Glider",
		[16184] = "Nerubian Overseer",

...into this?

code:

["FENROS"] = true,
["BLACKMOSS THE FETID"] = true,
["FENISSA THE ASSASSIN"] = true,
(and so on)

Vim regex is something like:

1,$ s/.*\(".*.").*/[\1] = true,/g
1,$ g/"/vi"gU

Jabor
Jul 16, 2010

#1 Loser at SpaceChem
Are you sure a regex is the right choice for your task? It seems unlikely that this is the only thing you want to do to that information, and doing it in your programming language of choice might make a lot of other things easier.

s/^.*(".*").*$/[\1] = true,/

Maha
Dec 29, 2006
sapere aude
It actually is the only thing I need to do with it, I'm just converting a database from one WoW addon to another.
And this is pathetic, but I've never done this before and can't figure out what to do in GVIM after loading the file. Could someone walk me through it?

e: nevermind, figured out a dumbass way to do it in notepad

Maha fucked around with this message at 23:22 on Dec 25, 2016

Polio Vax Scene
Apr 5, 2009



I am currently using an excel sheet to keep track of bugs in a project but it is getting pretty big and the way everything is in a table is getting unmanageable.
Looking for a piece of note entry software that meets these conditions.
- Free
- Online
- Able the view the list of entries, ordered based on status, date added/modified, etc.
- Automatically assigns numbers or IDs to the entries (optional)
- Image attachment (optional)
- Would be nice to be able to customize what data is needed for each entry but also optional.
The closest thing I've found is JIRA but I don't want to pay for it or set it up on a local server. I will be the only user, and the more simple/lightweight the better. The goal is something that I open in a browser and either a. type some bugs I found into notes to fix later or b. peruse the list of notes for things to fix now.
Please give me your best suggestions!

ultrafilter
Aug 23, 2007

It's okay if you have any questions.


How long can you count on being the only user?

Polio Vax Scene
Apr 5, 2009



Absolutely forever. Its for a personal project. If I ever do end up collaborating with someone, I would be willing to shell out for something like JIRA.

Doghouse
Oct 22, 2004

I was playing Harvest Moon 64 with this kid who lived on my street and my cows were not doing well and I got so raged up and frustrated that my eyes welled up with tears and my friend was like are you crying dude. Are you crying because of the cows. I didn't understand the feeding mechanic.
I think you can do that all in bitbucket iirc

Bob Morales
Aug 18, 2006


Just wear the fucking mask, Bob

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

Polio Vax Scene posted:

Absolutely forever. Its for a personal project. If I ever do end up collaborating with someone, I would be willing to shell out for something like JIRA.

There are like a hundred open source bug trackers out there.Trac, Redmine, Mantis...

nielsm
Jun 1, 2009



Bob Morales posted:

There are like a hundred open source bug trackers out there.Trac, Redmine, Mantis...

But hosting space is not free, unless you want to host out of your home internet.

dupersaurus
Aug 1, 2012

Futurism was an art movement where dudes were all 'CARS ARE COOL AND THE PAST IS FOR CHUMPS. LET'S DRAW SOME CARS.'
I like the issue tracker on Github, but that probably doesn't make sense if you're not using github (in which case you probably wouldn't be asking the question). JIRA is a big monster that's probably overkill if you're not doing corporate-scale work.

sarcastx
Feb 26, 2005



Quick question that I'm not sure where to ask:
I'm building a support panel in Powershell for our server/monitoring teams and need a way to launch Perfmon and immediately connect to a target server. Trouble is, I can't find the necessary flags to do so.
For example; I have a method to launch Services remotely by invoking "services.msc /computer=$target", and Event Viewer by invoking "eventvwr.exe $target", but I can't find a Perfmon flag that does the same for the life of me. Ideas?

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

I'm looking for reading, ideas, examples, or advice on writing fault tolerant web scrapers. Any suggestions?

pokeyman
Nov 26, 2006

That elephant ate my entire platoon.
Make it really easy to update the scraping rules whenever the page changes?

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

pokeyman posted:

Make it really easy to update the scraping rules whenever the page changes?

Yeah, I assume that's a pretty basic precaution to take. I'm just wondering about more automated ways of handling it. For example, what about if an element id you're depending on gets changed and the old id gets used on a different element? You might still be getting data but you don't realize it's the wrong data.

I'm sure there's not a great solution, just really wondering about all the issues and solutions people have come up with.


In other news....

I'm idly thinking about an Electron app that communicates with another local process over websockets. What things should I be thinking about with regards to security? I'm a hesitant about the idea of running a server on other users computers if only because it opens a path for potentially hostile other local processes into the users system.

Maluco Marinero
Jan 18, 2001

Damn that's a
fine elephant.
The problem with being defensive against change is that requires you to know what the nature of the change may be. If you don't the budgeted time is better spent by saving it for when the change is necessary.

The best way to be defensive in general so you get alerted when the scraper breaks is to have a validated schema for the final data, which may alert you to old order or tag assumptions no longer being valid. This doesn't protect if the change still matches the schema but is technically wrong, but again this goes back to understanding the nature of the potential changes.

Other than that, like what was said, make your code easy to alter when you do inevitably need to.

ulmont
Sep 15, 2010

IF I EVER MISS VOTING IN AN ELECTION (EVEN AMERICAN IDOL) ,OR HAVE UNPAID PARKING TICKETS, PLEASE TAKE AWAY MY FRANCHISE

Thermopyle posted:

Yeah, I assume that's a pretty basic precaution to take. I'm just wondering about more automated ways of handling it. For example, what about if an element id you're depending on gets changed and the old id gets used on a different element? You might still be getting data but you don't realize it's the wrong data.

Here you could have a specific known-good value to sanity-check against to make sure the scraper is still working before you look for new data.

sarehu
Apr 20, 2007

(call/cc call/cc)
Is it actually linear or is it m*n?

Polio Vax Scene
Apr 5, 2009



Doghouse posted:

I think you can do that all in bitbucket iirc

Thanks for this, bitbucket is doing great.

lifg
Dec 4, 2000
<this tag left blank>
Muldoon

Thermopyle posted:

I'm looking for reading, ideas, examples, or advice on writing fault tolerant web scrapers. Any suggestions?

I once built one that kept track of a control page with known content. If that page changed, the scraper looked for the known content, then found the nearest enclosing block, and used that new element to look for what to scrape in other pages.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Alright, thanks for the input on the scraper stuff. Pretty much what I had in mind. Was hoping someone would pop up with some great book or something that had a bunch of things I hadn't thought of.

To provide some content: I just discovered the Settings Repository plugin for Jetbrains products. This is super great as now I can keep my settings in sync between all my machines because they sync their settings to my github account. Hope someone finds this as useful as I do.

I mostly separate my dev projects and project types into different linux VMs running on my Windows host so now I can get around the nightmare of keeping my poo poo in sync.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Do any of these exception-capturing online tools like Sentry support attaching images like screenshots (or really any files) to the exception? I can't seem to figure out if Sentry does from briefly reading over the docs. I see that you can attach contextual data, but nothing specifically about images...

Sirocco
Jan 27, 2009

HEY DIARY! HA HA HA!
Is there a decent algorithm for generating distinct combinations (of a specific range) from lists with non-distinct elements which doesn't require filtering out duplicates afterwards?

For example if I had a list of ("a", "a", "b", "b") with a range of 3, I'd like to get ("a", "a", "b") and ("a", "b", "b") instead of ("a", "a", "b"), ("a", "a", "b"), ("a", "b", "b"), ("a", "b", "b").

baka kaba
Jul 19, 2003

PLEASE ASK ME, THE SELF-PROFESSED NO #1 PAUL CATTERMOLE FAN IN THE SOMETHING AWFUL S-CLUB 7 MEGATHREAD, TO NAME A SINGLE SONG BY HIS EXCELLENT NU-METAL SIDE PROJECT, SKUA, AND IF I CAN'T PLEASE TELL ME TO
EAT SHIT

You want to implement that algorithm, or just make that stuff happen in some language? A lot of them have library functions that will do at least some of the work, like filtering by distinct results

Sirocco
Jan 27, 2009

HEY DIARY! HA HA HA!
I'm working with Python but the itertools functions only provide unique results with unique values.

Hammerite
Mar 9, 2007

And you don't remember what I said here, either, but it was pompous and stupid.
Jade Ear Joe

Sirocco posted:

I'm working with Python but the itertools functions only provide unique results with unique values.

Without giving any thought to efficiency: you could use a list(range(len(your_list))) and translate by indexing into it elementwise once you have the answers from the library function, then filter for duplicates.

in other words, pass [0, 1, 2, 3] to the library function instead of ['a', 'a', 'b', 'b'] then apply the mapping 0 => 'a', 1 => 'a', 2 => 'b', 3 => 'b' so that e.g. (0, 1, 3) becomes ('a', 'a', 'b').

Eela6
May 25, 2007
Shredded Hen

Sirocco posted:

Is there a decent algorithm for generating distinct combinations (of a specific range) from lists with non-distinct elements which doesn't require filtering out duplicates afterwards?

For example if I had a list of ("a", "a", "b", "b") with a range of 3, I'd like to get ("a", "a", "b") and ("a", "b", "b") instead of ("a", "a", "b"), ("a", "a", "b"), ("a", "b", "b"), ("a", "b", "b").

baka kaba posted:

You want to implement that algorithm, or just make that stuff happen in some language? A lot of them have library functions that will do at least some of the work, like filtering by distinct results

This is the simplest thing that could work. Just use a set! I'm not sure if there's an 'actual' algorithm that doesn't involve membership testing at some point. If there is I'd love someone to point it out.

input:
Python code:
import itertools
def unique_n_combinations(iterable, n):

    unique_n_combinations = set(itertools.combinations(iterable, n))
    return unique_n_combinations
    # note: in python 3.5 and lower, the order of keys will not be preserved,
    # so if this is important you will want to sort afterwards
    
def test():
    my_list = ['a', 'a', 'b', 'b']
    three_combos = unique_n_combinations(my_list, 3)
    print(three_combos)
    assert three_combos == {('a', 'a', 'b'), ('a', 'b', 'b')}
output
code:

>>>test()
{('a', 'b', 'b'), ('a', 'a', 'b')}

Eela6 fucked around with this message at 17:02 on Jan 4, 2017

Sirocco
Jan 27, 2009

HEY DIARY! HA HA HA!
Thanks for the replies, but unfortunately the efficiency part is important. I'm trying to generate a list of all unique scrabble racks so I can store them in text files and then drag them out later to do operations on them. I already know how many there are (3199724 which is big, but not REALLY big) but actually creating the list of racks itself has been difficult because the duplicates bring the number of results up to over 16 billion which is unworkable computationally.

TooMuchAbstraction
Oct 14, 2012

I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.
Fun Shoe
3.2 million is big enough that I'd be seriously considering whether there are ways to accomplish my goals that don't require me to enumerate all of the racks. Can they be generated on an as-needed basis?

JawnV6
Jul 4, 2004

So hot ...

TooMuchAbstraction posted:

Can they be generated on an as-needed basis?

Yeah, AAAAAAA, AAAAAAB, AAAAABB, AAAAABC->AAAAABZ, AAAABBC, etc. Since you don't care about ordering, you just need a way increment from one valid sorted rack to the next.

The Laplace Demon
Jul 23, 2009

"Oh dear! Oh dear! Heisenberg is a douche!"

Sirocco posted:

Thanks for the replies, but unfortunately the efficiency part is important. I'm trying to generate a list of all unique scrabble racks so I can store them in text files and then drag them out later to do operations on them. I already know how many there are (3199724 which is big, but not REALLY big) but actually creating the list of racks itself has been difficult because the duplicates bring the number of results up to over 16 billion which is unworkable computationally.

Can't you pretend there's only 7 'a', 'e', 'i' and 'o' tiles? There are algorithms for directly computing the nth combination of a list. Nah, you'll need to write a bijection from the integers to the racks. Still should be able to do this without computing the previous racks.

Eela6
May 25, 2007
Shredded Hen
What are you actually trying to do?

Are you sure you need to have the text files? What kind of 'operations ' are we talking about?

Sirocco
Jan 27, 2009

HEY DIARY! HA HA HA!


I've been writing a little program for practising alphagrams for Scrabble. I can calculate the probability of drawing a specific rack but it doesn't tell you how common it is compared to other racks. So I was wanting to draw up something where I could, say, draw up the twenty most common racks because they'll merit more study. Many racks have the same probability so I know the number of possible probabilities is less than the 3.2 million racks. I thought it might be a good idea to generate a list of those instead and then map the letters onto it but I wasn't able to figure out how to do so.

nielsm
Jun 1, 2009



Instead of representing it as a list i.e. ('a', 'a', 'b'), represent it as a map with counts i.e. {'a': 2, 'b': 1}. Also represent the set of available tiles as a map with counts, or better as a list of (letter, count) pairs, to keep it ordered.
Then you can treat it as translating a rack combination index into a variable-radix number representing the rack.

E.g. for (('a', 3), ('b', 2), ('c', 2)) and 3-letter racks, you would map:

0 -> (3, 0, 0)
1 -> (2, 1, 0)
2 -> (2, 0, 1)
3 -> (1, 2, 0)
4 -> (1, 1, 1)
5 -> (1, 0, 2)
6 -> (0, 2, 1)
7 -> (0, 1, 2)

I'm not entirely sure how to implement the counting, but it is definitely possible to create a direct mapping like this.

E: You should also be able to calculate the probability of a specific rack after generating it, from the frequencies of each letter and the number of occurrences of it in the rack.

The Laplace Demon
Jul 23, 2009

"Oh dear! Oh dear! Heisenberg is a douche!"

Sirocco posted:

I've been writing a little program for practising alphagrams for Scrabble. I can calculate the probability of drawing a specific rack but it doesn't tell you how common it is compared to other racks. So I was wanting to draw up something where I could, say, draw up the twenty most common racks because they'll merit more study. Many racks have the same probability so I know the number of possible probabilities is less than the 3.2 million racks. I thought it might be a good idea to generate a list of those instead and then map the letters onto it but I wasn't able to figure out how to do so.

The only time this probability applies is during the initial draw, right? Most racks in a game are a mix of unused tiles and new ones. Brian Sheppard's thesis on Scrabble AI has a lot of discussion on human Scrabble technique that you might find interesting.

Nippashish
Nov 2, 2005

Let me see you dance!
This takes about a minute on my machine and generates the right number of racks:
code:
def _make_racks(n, letters, counts):
  if n == 0 or len(letters) == 0:
    yield []
  else:
    for i in range(min(counts[0], n) + 1):
      for end in _make_racks(n-i, letters[1:], counts[1:]):
        yield ([letters[0]] * i) + end

def make_racks(n, letters, counts):
  for rack in _make_racks(n, letters, counts):
    if len(rack) == n:
      yield rack

print(len(list(make_racks(7, letters, counts))))
letters is a list of letters and counts is the corresponding list of counts.

Adbot
ADBOT LOVES YOU

Eela6
May 25, 2007
Shredded Hen
You could use math to generate the probability of a particular rack appearing.

The probability is (# of ways to choose tiles that create our rack) / total 7-combinations from our choice of tiles.

Combinatorics is cool!


Python code:
from collections import Counter
from math import factorial

count = Counter({
    'e': 12, 'a': 9, 'i': 8,
     'o': 8, 'n': 6,'t': 6,
     'l': 4, 's': 4, 'u': 4,
     'd': 4 , 'g': 3,  'b': 2, 'h': 5,
     'c': 2, 'm': 2, 'p': 2,
     'k': 1, 'j': 1, 'x': 1, 
     'r': 13, 'v': 3, 'w': 4,
     'y': 4, 'q': 1, 'z': 1, 
     'f': 1, '*': 2
     })


        
def n_choose_k(n: int, k: int) -> int:
    """ choices of k objects from a collection of n"""
    if k > n:
        return 0
    return (factorial(n) // factorial(k) ) // factorial(n-k)

def copies_of_rack(rack: str) -> int:
    
    def product(iterable) -> int:
        total = 1
        for x in iterable:
            total *= x
        return total
        
    def ways_to_choose_copies_of_tile(tile: str) -> int:
        return n_choose_k(count[tile], rack_count[tile])
        
    rack_count = Counter(rack)
    copies = product(ways_to_choose_copies_of_tile(tile) for tile in rack_count)
    return copies
    
total_racks = sum(count[x] for x in count)

def frequency_of_rack(rack: str) -> float:    
    copies = copies_of_rack(rack)
    return copies / total_racks

Eela6 fucked around with this message at 06:44 on Jan 5, 2017

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply