|
What are my options for protecting strings in Python code from being easily memory dumped? Obfuscation doesn't help. Patched python built from source that does some transparent encryption? This is a product that will run on a dedicated machine that we can prepare, but will be installed on client premises. Of course it's not possible to completely prevent snooping but I'm not targeting NSA level stuff, just decent admins/programmers. Main problem is a lot of data used in complex computations is represented as strings (yay scientific stuff) and ideally shouldn't be possible to just be grepped from memory dump. Intel sgx is not an option unfortunately. I guess I'm hosed, but maybe I overlooked something?
|
#
?
Nov 27, 2020 21:22
|
|
|
|
| # ? May 18, 2024 13:20 |
|
|
If you're trying to hide super secret proprietary algorithms from competitors, I probably wouldn't put them in python. Write your secret stuff in C++ and compile a dll the Python code can call. It won't stop someone with Ghidra and a lot of free time, but it should be good enough.
|
|
#
?
Nov 27, 2020 21:28
|
|
|
Well yeah, I'm just trying to help someone who ships this in a month. Scientists love their python so that's what I've got.
|
|
#
?
Nov 27, 2020 21:32
|
|
|
Anything written in Python is easy to read. Nothing you can do to get around that.
|
|
#
?
Nov 27, 2020 22:02
|
|
|
e: nevermind. Sounds like you've got a problem that has "overly expensive 'enterprise' software solution" written all over it, if you aren't allowed to change how the scientists write anything.
Volmarias fucked around with this message at 01:06 on Nov 28, 2020 |
|
#
?
Nov 28, 2020 01:04
|
|
|
omeg posted:Well yeah, I'm just trying to help someone who ships this in a month. Scientists love their python so that's what I've got. Serve the python code as a web function and call the api with authentication. Maybe something like aws lambda and api gateway so it's scalable and secure?
|
|
#
?
Nov 28, 2020 04:17
|
|
|
Volmarias posted:e: nevermind. Sounds like you've got a problem that has "overly expensive 'enterprise' software solution" written all over it, if you aren't allowed to change how the scientists write anything. It can be changed, just probably not in the time frame I've got realistically if the changes are too drastic. There's a lot of academic grade code in there. Jerk McJerkface posted:Serve the python code as a web function and call the api with authentication. Maybe something like aws lambda and api gateway so it's scalable and secure? It is a web app but it will be deployed without internet access 
|
|
#
?
Nov 28, 2020 04:33
|
|
|
omeg posted:What are my options for protecting strings in Python code from being easily memory dumped? Obfuscation doesn't help. Patched python built from source that does some transparent encryption? This is a product that will run on a dedicated machine that we can prepare, but will be installed on client premises. Of course it's not possible to completely prevent snooping but I'm not targeting NSA level stuff, just decent admins/programmers. Main problem is a lot of data used in complex computations is represented as strings (yay scientific stuff) and ideally shouldn't be possible to just be grepped from memory dump. Intel sgx is not an option unfortunately. I guess I'm hosed, but maybe I overlooked something? We had a vmware OVA file (vm image) that we sent them, and then would pull down encrypted zip files with "docker" containers (our containers ran go and ruby code, but could easily run python) from a remote private server and run them on a "secure" linux machine, and did not give them the login details. In theory they could dump the running server's vmware memory to disk and then root around in there but that's an awful lot of work for little payout. They had also signed some paperwork before we gave them the OVA saying that they wouldn't do exactly that. The added benefit was that the machine could do "OTA updates" from behind a firewall, as it polled the private server, so as long as it wasn't totally airgapped, we could send them updates, although updating the "download and decrypt" scripts on the server remotely could get a little hairy on occasion, especially since the guy writing them notoriously never tested them before shipping them to customers, despite saying he did.
|
|
#
?
Nov 28, 2020 05:08
|
|
|
omeg posted:It can be changed, just probably not in the time frame I've got realistically if the changes are too drastic. There's a lot of academic grade code in there. It sounds to me like whatever security issues you think you have are probably dwarfed by the ones you actually have and don't know about.
|
|
#
?
Nov 28, 2020 06:47
|
|
|
Your best tool for protecting code deployed offline at client premises from being reverse-engineered or tampered with is an army of lawyers.
|
|
#
?
Nov 28, 2020 08:32
|
|
|
I think I've got a git issue (though I'm not sure it is actually an issue). Using GitHub, and we're using basically a git-flow model. We've got a develop and master branch. Branch off from develop, add a feature, PR, squash and merge into develop, then PR, squash and merge [possibly a few queued features] into master. Feature branches get deleted obviously, develop has a nice clean history of commits every time yadda yadda. Develop obviously never gets deleted, so on every merge to master, there is an ever-growing list of every previous commit. I've just ignored this as basically a UI issue: everything is fine, workflow works, I've become blind to this huge, expanding list of commits I get every time I PR against master. But my boss just questioned it, and I now think I'm possibly doing something dumb here. Is rebasing the only thing that's going to give me "clean" develop -> master PRs and is there any point doing that for the sake of nicer UI? Or is there likely something else stupid that I'm doing/not doing and I'm just missing something obvious?
|
|
#
?
Nov 30, 2020 16:27
|
|
|
We recently switched to an enforced linear history with rebase commits instead of merge commits, and it's a pain in the rear end, but gives us much better histories. We also switched to a monorepo, which I hated when it was suggested and still hate now. Basically, Git sucks and both merge commits and rebase commits have downsides. Pick the least-worst option for you.
|
|
#
?
Nov 30, 2020 16:56
|
|
|
Ach, I'm happy with that, as long as I'm not doing anything daft. I hate rebasing in situations that aren't just my personal local branch/es anyway, and I've got say on this, so merge commits it is. Heh, this is a monorepo as well, so that drastically adds to the number of commits, but hey ho. Was my choice to push for a monorepo in the first place though, and I'm pretty happy with it (and Yarn 2 to deal with it) compared to fannying on trying to glue together private git repos.
|
|
#
?
Nov 30, 2020 17:13
|
|
|
Git could benefit from hierarchical commits: From the perspective of the main project, you can keep a clean history by only looking at top-level commits. For individual PRs etc, you can keep your individual commits without losing them to a squash. Randall Munroe's description of git is the most elegant I've found:  Source: https://xkcd.com/1597/ Git could also benefit from a new, non-backwards-compatible CLI, and a new CLI command to delete and start fresh. Dominoes fucked around with this message at 17:18 on Nov 30, 2020 |
|
#
?
Nov 30, 2020 17:15
|
|
|
I would strongly consider only doing rebase commits on a monorepo. Merge commits are a great way to gently caress up someone else's project unintentionally.
|
|
#
?
Nov 30, 2020 17:15
|
|
|
Only rebase commits full stop? We have had a few very minor issues over the past year where work was wiped out, we'd just put it down to timing of branching/commit issues & human error, but thinking about it, would maybe have been avoided with rebasing. I'm possibly biased against rebase due to having a workflow a few years ago which was all rebasing, and that caused far more issues than I've ever had with just merge commits.
RobertKerans fucked around with this message at 17:45 on Nov 30, 2020 |
|
#
?
Nov 30, 2020 17:40
|
|
|
Yes, we only do rebase commits and the repo is configured to not let you push any merge commits (you'll get an error if you try). We've never had an issue with lost work. The only thing that comes up regularly is you'll open a PR and by the time it gets approved, master has been updated by someone else so you'll have to rebase again. Mildly annoying, but not a huge deal. The only really annoying thing is if you mess up and accidentally do a merge commit (or rebase the wrong direction), the repo won't let you just revert it - you have to hard reset. You learn not to do that quickly enough, though.
|
|
#
?
Nov 30, 2020 17:51
|
|
|
Dominoes posted:Git could benefit from hierarchical commits: From the perspective of the main project, you can keep a clean history by only looking at top-level commits. For individual PRs etc, you can keep your individual commits without losing them to a squash. That comic always breaks my heart. Git's CLI isn't doing it any favors, but it's not like the theoretical model is fundamentally unlearnable, and it's super useful once you do. People just...don't.
|
|
#
?
Nov 30, 2020 18:04
|
|
|
Git is a great example of something being really clean and elegant in theory, but then reality hits and it ends up with all sorts of crap tacked on to make it work for the specific things people want to do with it. I'm personally waiting for the inevitable 'disruption' to happen where everyone decides monorepos suck and microrepos are the ~~future~~.
|
|
#
?
Nov 30, 2020 18:08
|
|
|
KillHour posted:... make it work for the specific things people want to do with it. This is important. Git has several uses, that are used and overlap differently by user. Some examples: - Backing up your code, with the ability to re-examine arbitrary snapshots - Collaborating on projects with a tight team - Collaborating on projects with the public - Sharing code - Making code portable across multiple computers - A live, detailed changelog I think these differences cause some of the problems, and variety of experiences. Personally, I categorize a non-superficial understanding of Git with Haskell: Elegant in theory, but I've never been able to Grok it. It feels too... academic? Built by and for people who think in a different way than I'm accustomed to. That comic saved me recently: I was at my wit's end trying to squash a repo that was resulting in a cycle of errors. Deleted the .git folder, re-cloned master, and copy+pasted my files... Solved. There's always a way to fix problems with repos, but it often isn't worth the time and effort. Dominoes fucked around with this message at 18:24 on Nov 30, 2020 |
|
#
?
Nov 30, 2020 18:21
|
|
|
Dominoes posted:Personally, I categorize a non-superficial understanding of Git with Haskell: Elegant in theory, but I've never been able to Grok it. It feels too... academic? Built by and for people who think in a different way than I'm accustomed to. It was created by the kind of guy who makes his own OS "for fun" because none of the existing ones are ideologically pure enough. Theoretical beauty over practical reality is kind of the point. I use the GitHub Windows GUI thing and it is good enough for 99% of what I do with it.  Edit: Also, it was written specifically for managing the Linux kernel code. All the other stuff companies use it for weren't really in the original design, so don't think I'm blaming Linus for what it became. KillHour fucked around with this message at 18:30 on Nov 30, 2020 |
|
#
?
Nov 30, 2020 18:28
|
|
|
I use the git integration in VS code and that’s good for 99% of the stuff I need to do.
|
|
#
?
Nov 30, 2020 18:31
|
|
|
The Fool posted:I use the git integration in VS code and that’s good for 99% of the stuff I need to do. Can confirm; VS Code is the best thing Microsoft ever did after the Pipes 3D screensaver.
|
|
#
?
Nov 30, 2020 18:33
|
|
|
Another 2 points: A superficial understanding of git (ie memorize a subset of the commands to commit, push, pull etc) is often good enough, and there aren't many alternatives.
|
|
#
?
Nov 30, 2020 18:39
|
|
|
There's a difference between actually understanding how git works and having a rough mental model that means you're not just typing in magic words based on some set of rules. If I could get everyone to the point where they can at least make educated guesses about what to do in a situation they've never seen before I'd be very happy.
|
|
#
?
Nov 30, 2020 18:44
|
|
|
I just can't shake my hunch that the people who say things likeDominoes posted:A superficial understanding of git (ie memorize a subset of the commands to commit, push, pull etc) is often good enough and the people who lament getting their repos into unrecoverable states are largely the same people. It's good enough...until it isn't.
|
|
#
?
Nov 30, 2020 19:20
|
|
|
raminasi posted:I just can't shake my hunch that the people who say things like They are, but it's Git's fault for being opaque. It's way too easy to break something in a confusing way, and the terminology around all of it doesn't help.
|
|
#
?
Nov 30, 2020 19:30
|
|
|
I have a history kind of question. Outside of Scratch, has there ever been a gui based programming language? I kinda remember a early version of VB.net having a gui that you could use to make programs and apps
|
|
#
?
Nov 30, 2020 19:45
|
|
|
KillHour posted:It was created by the kind of guy who makes his own OS "for fun" because none of the existing ones are ideologically pure enough. Theoretical beauty over practical reality is kind of the point. That is literally the exact opposite of Linux. The Tanenbaum-Torvalds debate centered around the fact that Linux was a hack job which focused on building something that worked with no regard to architectural cleanliness or long-term design.
|
|
#
?
Nov 30, 2020 19:45
|
|
|
Empress Brosephine posted:I have a history kind of question. Outside of Scratch, has there ever been a gui based programming language? I kinda remember a early version of VB.net having a gui that you could use to make programs and apps I mean, there's quite a few: https://en.wikipedia.org/wiki/Visual_programming_language They're mostly in the business process space as the scope is smaller and the users are less technical.
|
|
#
?
Nov 30, 2020 19:49
|
|
|
Thanks. I had no idea although I remember messing with Construct. Visual Basic was what I was thinking of. That was a fun tool as a 15 year old
|
|
#
?
Nov 30, 2020 19:55
|
|
|
Empress Brosephine posted:I have a history kind of question. Outside of Scratch, has there ever been a gui based programming language? I kinda remember a early version of VB.net having a gui that you could use to make programs and apps I remember helping my college roommate do some bio-engineering homework in LabVIEW (and being deeply disturbed by it).
|
|
#
?
Nov 30, 2020 20:03
|
|
|
KillHour posted:They are, but it's Git's fault for being opaque. It's way too easy to break something in a confusing way, and the terminology around all of it doesn't help. I genuinely do not understand this sentiment, using git for something like the github flow workflow is extremely simple and intuitive. The magic words that you start by googling and using map directly to what git is actually doing 1:1, which allows you to build deeper understanding for things that actually are not intuitive (interactive staging, cherry-pick) slowly over time. As someone new to git, it is definitely easy to get into a state where you can't do what you want to do, but it's really hard to actually _break_ something, especially in a way that isn't recoverable. Dominoes posted:I was at my wit's end trying to squash a repo that was resulting in a cycle of errors. Deleted the .git folder, re-cloned master, and copy+pasted my files... Solved. There's always a way to fix problems with repos, but it often isn't worth the time and effort. You probably already know this, but for any other thread readers and so long as you can still check out branches on your repository, you can just check your files out from wherever you last put (committed) them. For example, if you have branch feature-1 and it's, for some reason, "broken", just branch a new feature-1a from main and "git checkout feature-1 ." You include the dot for "all of the files from this location", just like when you want to stage all of your local changes you run "git add ." -- and, like both, you can check out individual files or add individual files. Git's syntax for the basic core commands is very internally consistent. If you haven't committed your files to your branch yet, you can just stash them: "git stash". When you want them again, for example because you are on a different branch, "git stash pop". IME a lot of the problems people have with git start when they don't build this mental model of how it actually works, and they do google and consult stack overflow, which is full of people posting like full awk scripts and crap for extremely basic git one liners. 12 rats tied together fucked around with this message at 20:43 on Nov 30, 2020 |
|
#
?
Nov 30, 2020 20:37
|
|
|
Empress Brosephine posted:I have a history kind of question. Outside of Scratch, has there ever been a gui based programming language? I kinda remember a early version of VB.net having a gui that you could use to make programs and apps Excel gets closer to one with every release
|
|
#
?
Nov 30, 2020 20:41
|
|
|
Munkeymon posted:Excel gets closer to one with every release Excel is already Turing complete and so is PowerPoint, for that matter.
|
|
#
?
Nov 30, 2020 20:46
|
|
|
Empress Brosephine posted:I have a history kind of question. Outside of Scratch, has there ever been a gui based programming language? I kinda remember a early version of VB.net having a gui that you could use to make programs and apps The magic software (https://www.magicsoftware.com/) people make one, called either "magic xpa" or "magic unipaas", I can't remember which. I used to work IT at a place that used this, if I had to guess it's basically a really fancy windows forms wrapper that can technically run as a web service. As a rapid application development thing it's actually not bad -- the place I worked tried, and failed, to do a magic -> c#/.net rewrite and ended up failing because our 2 magic developers were doing the work of 40 c# developers. From an operations perspective the thing was a piece of hot garbage and our entry level IT role involved an hourly timer where you would manually log onto each "magic server" and check the interface, which ran as a program that required an active login session, to make sure that it was doing things.
|
|
#
?
Nov 30, 2020 20:55
|
|
|
12 rats tied together posted:but it's really hard to actually _break_ something, especially in a way that isn't recoverable. 12 rats tied together posted:You probably already know this, but for any other thread readers and so long as you can still check out branches on your repository, you can just check your files out from wherever you last put (committed) them. For example, if you have branch feature-1 and it's, for some reason, "broken", just branch a new feature-1a from main and "git checkout feature-1 ." quote:IME a lot of the problems people have with git start when they don't build this mental model of how it actually works, and they do google and consult stack overflow, which is full of people posting like full awk scripts and crap for extremely basic git one liners. Dominoes fucked around with this message at 21:11 on Nov 30, 2020 |
|
#
?
Nov 30, 2020 21:08
|
|
|
Also, does TIBCO StreamBase count?
|
|
#
?
Nov 30, 2020 21:10
|
|
|
12 rats tied together posted:I genuinely do not understand this sentiment, using git for something like the github flow workflow is extremely simple and intuitive. Now imagine someone who doesn't find it extremely simple and intuitive.
|
|
#
?
Nov 30, 2020 21:11
|
|
|
|
| # ? May 18, 2024 13:20 |
|
|
You might not be able to break anything permanently in a git repo, but you can definitely lose local changes that you haven't pushed to a remote.
|
|
#
?
Nov 30, 2020 21:22
|
|