Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
The Fool
Oct 16, 2003


I made a command line tool in C# that solves a specific problem and want to put it up on github.

This is the first time I've created something that I want to share with other people, what things should I do to be a good open source citizen besides just writing up a readme.md and throwing it up there?

Adbot
ADBOT LOVES YOU

nielsm
Jun 1, 2009



Decide on a license and make sure it's attached correctly to the code.

goodness
Jan 3, 2012

When the light turns green, you go. When the light turns red, you stop. But what do you do when the light turns blue with orange and lavender spots?

The Fool posted:

I made a command line tool in C# that solves a specific problem

What does it do/whats the problem

pokeyman
Nov 26, 2006

That elephant ate my entire platoon.

The Fool posted:

I made a command line tool in C# that solves a specific problem and want to put it up on github.

This is the first time I've created something that I want to share with other people, what things should I do to be a good open source citizen besides just writing up a readme.md and throwing it up there?

Make a decent readme and you’re most of the way there. It should say what your thing does, why someone would use it, how to install it, how to use it, and how to clone the repo and build it. Add a couple of usage examples, the first of which is the simplest possible useful invocation.

Turn off the Issues tab if you don’t plan to respond to opened issues. (It is entirely ok to do this, and it’s better than having a half dozen five-year-old unanswered issues hanging around.)

If it’d be useful to have a compiled binary for download, bump the version and make a git tag when you build one, then add the binary to the tag in the Releases page (and link to Releases in your readme). (This is where the build instructions in your readme come in handy: when you forget all this and want to ship a lil update in six months.)

And yeah as mentioned pick a license and mention it in the right place.

The Fool
Oct 16, 2003


Thanks for the advice, now to write a decent readme

goodness posted:

What does it do/whats the problem

Exports user audit history from Dynamics CRM online to a csv

Xarn
Jun 26, 2015
I disagree with turning off issues -- being able to quickly see through problems other people ran into when using a tool is valuable, even if the original author doesn't plan to ever respond.

pokeyman
Nov 26, 2006

That elephant ate my entire platoon.

Xarn posted:

I disagree with turning off issues -- being able to quickly see through problems other people ran into when using a tool is valuable, even if the original author doesn't plan to ever respond.

This can be nice, I agree. I wanted to make it clear that being a "good open source citizen" doesn’t have to involve unlimited free prompt support.

Personally I leave issues turned on, but I enjoy responding to people and it doesn’t feel like a chore to me. If it ever did start being a pain in my rear end, I’d turn that poo poo off without hesitation.

AgentCow007
May 20, 2004
TITLE TEXT
What is the point of the compsci course "automata"? I just finished the class with a B and I don't have the faintest idea what it was or how it would be applied to anything. I got that regex was briefly covered, and some other things that were like regex, but that's about it. At what point in one's programming career might one be like "it's time to bust out some automatas"?

Thanks for the help, you guys have been invaluable for my "what is the point of x" compsci questions.

nielsm
Jun 1, 2009



AgentCow007 posted:

What is the point of the compsci course "automata"? I just finished the class with a B and I don't have the faintest idea what it was or how it would be applied to anything. I got that regex was briefly covered, and some other things that were like regex, but that's about it. At what point in one's programming career might one be like "it's time to bust out some automatas"?

Thanks for the help, you guys have been invaluable for my "what is the point of x" compsci questions.

The idea is having a formal description of what a state machine is, so you can recognize one when you see it, and have (mental) tools to work with them and reason about them. State machines are everywhere, but if you don't know they are a thing and don't know there is formal theory for working with them, you have much greater risk of writing bad code when working on one, whether you know it is one or not.

State machines are typical in e.g. network protocols, ERP systems, game logic, pretty much anything will have them at some level or other.

nielsm fucked around with this message at 09:04 on Dec 22, 2018

dirby
Sep 21, 2004


Helping goons with math

AgentCow007 posted:

What is the point of the compsci course "automata"?

Also it's a compsci course, not a programming course, so it need not have any direct bearing on programming you do.

Similarly, the Halting Problem is very significant in compsci but isn't going to affect programming in a direct way, etc.

dirby fucked around with this message at 15:43 on Dec 22, 2018

Bruegels Fuckbooks
Sep 14, 2004

Now, listen - I know the two of you are very different from each other in a lot of ways, but you have to understand that as far as Grandpa's concerned, you're both pieces of shit! Yeah. I can prove it mathematically.

AgentCow007 posted:

What is the point of the compsci course "automata"? I just finished the class with a B and I don't have the faintest idea what it was or how it would be applied to anything. I got that regex was briefly covered, and some other things that were like regex, but that's about it. At what point in one's programming career might one be like "it's time to bust out some automatas"?

Thanks for the help, you guys have been invaluable for my "what is the point of x" compsci questions.

Automata theory is part of the Theory of Computation, which you can divide into three categories - automata theory and languages, computability theory, and computational complexity theory. The basic theme of the Theory of Computation is "What are the fundamental capabilities and limitations of computers?"

As for practical uses for automata, well aside from regex, you can use bison (https://www.gnu.org/software/bison/) to create a parser for your programming language. They're used pretty heavily in compilers and language processing and AI. There's also the state machine pattern, and UML modeling is essentially modelling a system as a state machine.

ultrafilter
Aug 23, 2007

It's okay if you have any questions.


Variants of automata are useful for modeling systems that have some property. You will need to be very comfortable with automata theory if you want to do real time computing or robotics.

On the less academic side, if you're designing a user interface with any degree of complexity, you're going to suffer if you don't model it as a state machine.

Somaen
Nov 19, 2007

by vyelkin
Hello, I have a basic idea of programming and only vague ideas of libraries, depositories, github, etc. I want to create a project in Python where a program would would track some simple camera input, like a tree rustling in the wind, and draw a sinusoidal function based on how far some specific pixels move from equilibrium.

What would my first steps be? Just googling? What are some repositories of libraries that people normally turn to?

Crack
Apr 10, 2009
On macos using launchd when launching a python file why does "#! /usr/local/bin/python3" work but "#! /usr/bin/env python3" not? I get "env: python3: No such file or directory" when using the latter but they both work fine from the command line.

nielsm
Jun 1, 2009



Somaen posted:

Hello, I have a basic idea of programming and only vague ideas of libraries, depositories, github, etc. I want to create a project in Python where a program would would track some simple camera input, like a tree rustling in the wind, and draw a sinusoidal function based on how far some specific pixels move from equilibrium.

What would my first steps be? Just googling? What are some repositories of libraries that people normally turn to?

First step would be researching computer vision libraries you can use for the image recognition. Use Google or whatever search engine for that. Pick one that looks promising and see if you can make it recognize your scene and measure the data you're interested in. If you find more than one, it might be an idea to try using several of them, see which works better for you. After making a smallest-viable proof of concept, you can start worrying about the infrastructure.

When you're not experienced I think it's better to focus on the core problem first, before worrying about auxiliary things like project structure, dependency management, packaging, and so on.

TooMuchAbstraction
Oct 14, 2012

I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.
Fun Shoe

Somaen posted:

Hello, I have a basic idea of programming and only vague ideas of libraries, depositories, github, etc. I want to create a project in Python where a program would would track some simple camera input, like a tree rustling in the wind, and draw a sinusoidal function based on how far some specific pixels move from equilibrium.

What would my first steps be? Just googling? What are some repositories of libraries that people normally turn to?

Research image processing techniques. I believe specifically that cross-correlation and Fourier transforms would be useful for what you want to do -- you may be able to cross-correlate a frame with the previous frame to get the offset you want. For Python, the numpy and scipy libraries will probably have the tools you need, but without the knowledge on how to use the tools that doesn't help you much.

pokeyman
Nov 26, 2006

That elephant ate my entire platoon.

Crack posted:

On macos using launchd when launching a python file why does "#! /usr/local/bin/python3" work but "#! /usr/bin/env python3" not? I get "env: python3: No such file or directory" when using the latter but they both work fine from the command line.

The latter searches the path for a binary called python3, and it sounds like launchd is running as a user or in a session that does not have /usr/local/bin in the PATH environment variable. The former doesn’t consult PATH at all so it works fine. And in your terminal session you have a different PATH with /usr/local/bin listed, so it works ok for you.

If you really want to consult PATH in your shebang, you’ll have to find a way to add /usr/local/bin to PATH in your launchd plist (or find some way to add it before it runs your python file).

Crack
Apr 10, 2009
Thanks, that makes sense. I thought that plists in my home library would be run as me, but there wasn't much behind that assumption.

As a somewhat related question, I made a very simple script to open a text file every fifteen minutes as a reminder:

code:
import subprocess

subprocess.Popen(['open', '/Applications/TextEdit.app', '/Users/crack/Script Files/reminder.txt'])
This again works fine from the command line, but it doesn't open the file and exits without error. I suspect based on the previous answer it's being run sucessfully by the launchd user / session and not me, but is it possible to have it run as me or at least displayed to me like that?

e: maybe this isn't the problem, I used os.geteuid() and os.getuid() and it is actually running as me.

e2: figured this out in the end, AbandonProcessGroup needed to be set to True.

Crack fucked around with this message at 01:14 on Dec 24, 2018

fritz
Jul 26, 2003

Somaen posted:

Hello, I have a basic idea of programming and only vague ideas of libraries, depositories, github, etc. I want to create a project in Python where a program would would track some simple camera input, like a tree rustling in the wind, and draw a sinusoidal function based on how far some specific pixels move from equilibrium.

What would my first steps be? Just googling? What are some repositories of libraries that people normally turn to?

Just to add on to what's been said, you might be interested in optical flow : https://en.wikipedia.org/wiki/Optical_flow

LongSack
Jan 17, 2003

Ok, I’m not much of an algorithms guy, and I’m wondering if there’s a good way to handle this problem.

I am working on a tool which will read the spreadsheets that users submit to my team when requesting new firewall rules and produce a script that can be fed into a firewall to create the rules. The spreadsheets are in a standard format, and basically list the source address, destination address, protocol and port(s). Oh and any of those can be an object group, which I haven’t even started thinking about yet.

My program reads the saved firewall configurations from a TFTP server and models each firewall, so I know their connected interfaces, routing tables (thankfully, it’s all static routes), defined objects and object groups, and existing rules (which will be used for duplicate rule detection).

My tool produces an object for each line in the spreadsheet which contains the source and destination firewalls, the ingress and egress interface for each, and some other info. It then collapses any rules for the same source/dest/protocol tuple into a single source object with all the ports concatenated.

Here is my problem (sorry for all the :words:). I want to check if the list of ports can be completely or partially replaced by an existing object group. I have the exact match working fine, so now I want to tackle partial matches.

So, given a set of ports P, I want to produce a collection of zero or more candidates C1..n with the following criteria:
  • each candidate must be a subset of P (can’t open more ports than were requested)
  • the best is not necessarily the single group with the longest match, but the combination of groups with the longest match. For example, if P contains 6 ports, two groups which together cover 5 of them is better than one which covers 4.
  • overlap of ports among the candidates is not ideal, but acceptable
  • there can be (and almost certainly are) duplicate groups. It makes no difference which of the duplicates is selected, be it first, last, or whatever
I’m thinking something with set theory but am clueless as to how to do it (short of brute force).

If it matters, the code is in C#.

I do have Sedgewick’s Algorithms (Fourth Edition).

Anyone have any likely starting points? TIA

rarbatrol
Apr 17, 2011

Hurt//maim//kill.

dirby posted:

Similarly, the Halting Problem is very significant in compsci but isn't going to affect programming in a direct way, etc.

Every now and then, I have to explain it to a PM who wants to add resiliency to the product.

rarbatrol fucked around with this message at 05:19 on Dec 24, 2018

downout
Jul 6, 2009

AgentCow007 posted:

What is the point of the compsci course "automata"? I just finished the class with a B and I don't have the faintest idea what it was or how it would be applied to anything. I got that regex was briefly covered, and some other things that were like regex, but that's about it. At what point in one's programming career might one be like "it's time to bust out some automatas"?

Thanks for the help, you guys have been invaluable for my "what is the point of x" compsci questions.

Based on other posts, this sounds like a class I took, but it wasn't called "automata". That said I also did not have the faintest idea wth was going on, and also passed with a B. I find it important when debugging. It allows me to compartmentalize various sections of a program and treat them as abstract parts. That type of thing really helps me find where the failure is happening or even find where something should "split up" so that it will be easier to identify failures. It's an incredibly abstract consideration of what programmers do every day, and I couldn't say anything more without sounding like an idiot. But I've often found developers that have not been formally trained to be completely poo poo in regards to concepts that class made me appreciate. I still don't understand them, but I bet if I went on for my PHD it would probably be a very big "oh now I get it" moment for me. Maybe.

Dren
Jan 5, 2001

Pillbug

rarbatrol posted:

Every now and then, I have to explain it to a PM who wants to add resiliency to the product.

“Why can’t the compiler verify that this function won’t throw any exceptions that aren’t declared as being thrown? It should do that!”

eschaton
Mar 7, 2007

Don't you just hate when you wind up in a store with people who are in a socioeconomic class that is pretty obviously about two levels lower than your own?

dirby posted:

Similarly, the Halting Problem is very significant in compsci but isn't going to affect programming in a direct way, etc.

Until someone tries to get you to write a tool that, given an arbitrary PostScript file, does mail merge on it while preserving formatting including centering and justification.

eschaton
Mar 7, 2007

Don't you just hate when you wind up in a store with people who are in a socioeconomic class that is pretty obviously about two levels lower than your own?

Crack posted:

On macos using launchd when launching a python file why does "#! /usr/local/bin/python3" work but "#! /usr/bin/env python3" not? I get "env: python3: No such file or directory" when using the latter but they both work fine from the command line.

You need to pass the full path to python3, when launchd tries to run the executable you specify it does so with a very limited PATH.

CPColin
Sep 9, 2003

Big ol' smile.

LongSack posted:

I’m thinking something with set theory but am clueless as to how to do it (short of brute force).

You could always start with the brute force approach and go from there if you need it to be faster. For N candidates, you have 2N combinations to check, which shouldn't take too long.

goodness
Jan 3, 2012

When the light turns green, you go. When the light turns red, you stop. But what do you do when the light turns blue with orange and lavender spots?
I'm reviewing Arrays and got a confusion concept, 4+ dimensional arrays. Is there anyway to visualize these and what is an example of an array that would need to be 4+ dimensions?

RPATDO_LAMD
Mar 22, 2013

🐘🪠🍆
It's like a 3d array but with one more dimension.
In the same way that a 2d array is just a bunch of 1d arrays sitting next to each other in a row, a 4d array is just a bunch of 3d arrays next to each other. Same for 5d etc.
In the end a 200-dimensional array is not really any more complex to work with than a 2-dimensional array.

baka kaba
Jul 19, 2003

PLEASE ASK ME, THE SELF-PROFESSED NO #1 PAUL CATTERMOLE FAN IN THE SOMETHING AWFUL S-CLUB 7 MEGATHREAD, TO NAME A SINGLE SONG BY HIS EXCELLENT NU-METAL SIDE PROJECT, SKUA, AND IF I CAN'T PLEASE TELL ME TO
EAT SHIT

I just think of like one element as a box, and a simple 1D array is a line of boxes. A 2D array is a square of boxes, where you've extruded each box in your original line into its own line - each element contains its own array of elements

If you want another dimension - stretch each of those boxes out so you get a cube, now it's 3D. Need another? Have a line of those cubes, now it's 4D. Add another dimension and you have a grid of cubes, etc etc

If you visualise this you kinda end up with structures inside structures, so it might end up being easier to think of array elements containing references to another array that's somewhere else, which might itself contain references... depends how deep you're nesting this stuff. And I've talked about squares and cubes, but they could be arbitrary lengths, so that was more for a simple visual picture

But another way to think of it, is you have a drawer with a few folders in it. That's sort of an array! Maybe the drawer is divided into a few sections, each with their own set of folders. Now it's a 2D array. There's multiple drawers in a cabinet, now it's 3D - folders inside sections inside drawers. And you can keep going with this - a line of cabinets, several lines of cabinets in a room, a corridor of rooms, multiple corridors, floors of corridors, buildings... It's not that complicated, just there's a lot of stuff - but you could easily and specifically reference any element of this because of how it's organised

You could arrange any data that way, although it's often more convenient to use a more readable structure instead of stringing indices together and expecting the reader to know what each layer means. But sometimes that's easier too, and arrays tend to be more performant (not sure about Python specifically)

asur
Dec 28, 2012

goodness posted:

I'm reviewing Arrays and got a confusion concept, 4+ dimensional arrays. Is there anyway to visualize these and what is an example of an array that would need to be 4+ dimensions?

Any 3d array and the addition of time is a common example of a 4d array.

SurgicalOntologist
Jun 17, 2004

Yeah, like temperature over time in a 3D space: x, y, z, t. A common data structure in meteorology (I imagine).

I use high dimensional arrays for 2D trajectories with dimensions >2 as categorical variables identifying the entity.

goodness
Jan 3, 2012

When the light turns green, you go. When the light turns red, you stop. But what do you do when the light turns blue with orange and lavender spots?
All of those are great examples that helped me visualize it. Thank you.

The temperature over time in a 3d space was especially helpful as a real world example.

stray
Jun 28, 2005

"It's a jet pack, Michael. What could possibly go wrong?"
Is it possible to use programmatic means (I'm a Python person) to block Facebook pages? I decided to go through my FB ad prefs today and it's almost exclusively car dealerships and realtors. I'm not looking for a house or a new car, so I don't know why they're there. I also don't want these chuckleheads ending up back in my ads, so I'd like to block them all, but there are hundreds of them in my list. (That screenshot isn't even all of them!) Is there a way to mass-block them?

huhu
Feb 24, 2006

stray posted:

Is it possible to use programmatic means (I'm a Python person) to block Facebook pages? I decided to go through my FB ad prefs today and it's almost exclusively car dealerships and realtors. I'm not looking for a house or a new car, so I don't know why they're there. I also don't want these chuckleheads ending up back in my ads, so I'd like to block them all, but there are hundreds of them in my list. (That screenshot isn't even all of them!) Is there a way to mass-block them?

Adblock? Or https://www.raspberrypi.org/blog/pi-hole-raspberry-pi/

goodness
Jan 3, 2012

When the light turns green, you go. When the light turns red, you stop. But what do you do when the light turns blue with orange and lavender spots?

stray posted:

Is it possible to use programmatic means (I'm a Python person) to block Facebook pages? I decided to go through my FB ad prefs today and it's almost exclusively car dealerships and realtors. I'm not looking for a house or a new car, so I don't know why they're there. I also don't want these chuckleheads ending up back in my ads, so I'd like to block them all, but there are hundreds of them in my list. (That screenshot isn't even all of them!) Is there a way to mass-block them?

Ublock origin. Blocks all the ads on Facebook, YouTube, etc. I havent seen an ad on a browser in 5+ years

Walh Hara
May 11, 2012
I just started working in a newly created data engineering team, we're (helping with) implementing data science models. I.e. converting jupyter notebooks made by a data scientist into a daily script on a server. We're looking for documentation on best practices about this. I.e. design guidelines. We work with python, hadoop, pyspark, jenkins, pybuild.

Practical questions we have:
- how should we organise logging? How do we make it manageable?
- how should we work with configuration and parameters. I.e. should the location of input parquet files be hardcoded in the script, put in a config.ini or read via the command line arguments?
- how should we build/ install python code? Topics we want to learn about include source distribution, binary distribution, wheel, egg,...
- other guidelines and best practices...

Anyone can recommend some good resources for these questions?

CPColin
Sep 9, 2003

Big ol' smile.

stray posted:

Is it possible to use programmatic means (I'm a Python person) to block Facebook pages? I decided to go through my FB ad prefs today and it's almost exclusively car dealerships and realtors. I'm not looking for a house or a new car, so I don't know why they're there. I also don't want these chuckleheads ending up back in my ads, so I'd like to block them all, but there are hundreds of them in my list. (That screenshot isn't even all of them!) Is there a way to mass-block them?

Open up whichever tab of the Ad Preferences page has stuff listed, click "See More" a few times to get a good number showing, and run this statement in your browser's developer console:

JavaScript code:
document.querySelectorAll("button[title=Remove]").forEach(button => button.click())
Your browser may hang for a few seconds as the big pile of HTTP POST requests goes through. This should work until the next few dozen realtors somehow get your contact info.

goodness
Jan 3, 2012

When the light turns green, you go. When the light turns red, you stop. But what do you do when the light turns blue with orange and lavender spots?
What are some terms I should google to learn about coding a program to Take a video file of a tv show, connect its unformatted title like (Battlestar.Galactica.(2003).S01.E01.1080p.Bluray.AAC.5.1.x265-LION[UTR] to its series using an API call, and then change the title to a new format like <Show - Title - Year>?

I'm decent at C++ now and starting python.

Mr Shiny Pants
Nov 12, 2012

goodness posted:

What are some terms I should google to learn about coding a program to Take a video file of a tv show, connect its unformatted title like (Battlestar.Galactica.(2003).S01.E01.1080p.Bluray.AAC.5.1.x265-LION[UTR] to its series using an API call, and then change the title to a new format like <Show - Title - Year>?

I'm decent at C++ now and starting python.

HTTP requests, HTTP request parsing, OS file operations, string formatting and depending on the API maybe some JSON parsing.

Adbot
ADBOT LOVES YOU

baka kaba
Jul 19, 2003

PLEASE ASK ME, THE SELF-PROFESSED NO #1 PAUL CATTERMOLE FAN IN THE SOMETHING AWFUL S-CLUB 7 MEGATHREAD, TO NAME A SINGLE SONG BY HIS EXCELLENT NU-METAL SIDE PROJECT, SKUA, AND IF I CAN'T PLEASE TELL ME TO
EAT SHIT

Regular expressions seem like a good match for that? I'm guessing you have a bunch of inconsistent filenames, so you want a general rule for pulling out the bits of info you want, which are probably something like

  • The series title is at the start of the filename (maybe ignoring any leading symbols)
  • The year is 4 digits in parentheses, following the title
  • Series and episode are Sxx and Exx where those X's are digits
  • Everything else is trash?

You could write a regex that pulls those out, and do any processing you need on those bits (like fixing title spaces). You could use normal searching logic in your language, but regex can express a lot fairly simply. Depends how much variation you expect to see in your title formatting, and how much you want to handle instead of saying "can't do anything with this one"

regex101 is good to play around and test things

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply