|
Is it possible to transfer a an Atlassian product license (looking at JIRA+Stash+Bamboo) between machines? I'm looking at getting a personal license, I'd like to throw it on my server bare-metal, but if it's gonna be bitchy about getting moved I'll toss it in a VM instead. Sorry, not quite on the thread topic, just seemed like people here might have some answers 
Paul MaudDib fucked around with this message at 03:21 on Feb 28, 2015 |
#
¿
Feb 28, 2015 02:50
|
|
|
|
| # ¿ Apr 29, 2024 02:32 |
|
|
I'd create a repo with a git branch/tag for each environment config. You have something that says "docker: bring up server with configuration 'production'" and then it does "git archive --format=tar.gz git://gitconfigrepo:production | tar -xz". I can't remember the exact syntax to archive only a specific branch or tag, but I wrote it up as a script a while back. I'll check on that. There's probably a better mechanism for making the config update atomically when you change the syntax to prevent a timespan where configs are broken, maybe subtrees or submodules? But I think that works for a simple deployment scheme.
|
|
#
¿
May 28, 2015 07:01
|
|
|
minato posted:We had branch-per-configtype for a while (one for prod, dev, staging, test, etc), and it went really badly because it turns out that many changes frequently have to be made to multiple branches simultaneously (e.g. updating what SSL ciphers an Apache instance uses), and the branches drift apart from each other. That is a better solution and I'll keep that in mind.
|
|
#
¿
May 29, 2015 07:32
|
|
|
a worthy uhh posted:10 separate "Buildings" Be honest with me here, are you delivering medical services? Customized config on both a facility and room level, airgapped system, maybe like a cyclotron treatment facility or something? hire me esp. if you know Thornton I used to hear horror stories about cyclotrons and cobbled-together bullshit in clearcase. Including one about a guy who had someone bring in a scorched plexiglas testing cube and drat near shat himself when he realized that it was nuking the cube. I'm honestly amazed there's never been another Therac. If that's what you're doing make goddamn sure that that can never ever happen. Especially with continuous integration. Paul MaudDib fucked around with this message at 06:34 on Jun 4, 2015 |
|
#
¿
Jun 4, 2015 04:55
|
|
|
I have also sold our team on continuous deployment to staging, at least. We still don't have any goddamned testing because my predecessors were retards but I have permission to start fixing it going forward, just like the rest of the codebase. I think I may be getting promoted to the senior engineer on the project, my boss gave me a nice talk about how much I have improved "process" on the project and talked about how we needed someone to make strategic decisions and onboard newbies. We have a fuckload of supported browsers and OSs, an unsustainable amount if humans have to monitor the differences. We obviously need unit tests too, but as a way to automate regression testing, I'm strongly considering implementing a visual diff tool with an adjustable alert threshold type thing that monitors difference from a baseline browser as well as version over version change. I think that would be pretty straightforward and would give us a big red flag when something breaks. Paul MaudDib fucked around with this message at 05:26 on Jun 4, 2015 |
|
#
¿
Jun 4, 2015 05:19
|
|
|
a worthy uhh posted:RE: Github, When you say publicly available, does that mean it's hosted locally, or on their servers but private? We also have a requirement that nothing can be hosted by a third party (which is horrible but it comes with the territory). This also addresses VS Online, which I assume hits the same roadblock. I'll check them both out for suitability though, thank you both. Hosted or solely hosted? Git is a distributed version system, every developer holds a full copy of the repo. If Github fucks you over any one of them is capable of restoring you all on their own. Put it on a backed up system and you're golden. If you mean your legal has a problem with someone else having access to your source (i.e. you have a problem with them hosting it, at all), that's also bullshit. If Github disclosed someone's source just for funsies the repercussions would put them out of business in a week. Hate to put it quite like this but Github is a Facebook-level business, your software isn't worth loving it up over. Even if you're a whole complex of highly profitable medical facilities. There's much, much bigger fish than you who are retaining their services. Get your boss to stand ground on these issues, neither are acceptable complaints. PHI isn't the same thing at all - doesn't need to go into the repo. We deal with PHI and we don't do it. Keep that poo poo on an audit-logged server. Our reporting is nuts, I have actually been considering re-vamping our audit logging into a separate service (DB + macaroons) since what we have now is incompatible with our current system layout. Right now we can't guarantee that someone wouldn't start a transaction, do queries, and then roll the audit log back. It's not a "yesterday" issue because our network is super isolated from the net, so there's like 2-3 people who could do that. It works in staging b/c a junior dev rolled back audit logging and then built functionality on top of the concept of being able to perform un-loggable SELECT queries on PHI (he's being fired next week). I'm actually going to muscle that issue again tomorrow, we have a team that's building on that functionality for a release in 2-3 weeks and I really cannot allow that to deploy into prod. syphon posted:I'd probably tackle your challenges in this order: This is really good advice - this is what I've been doing with Bamboo. Check out the "git-flow" model - that's the most sensible model I've seen so far. "Develop" is equivalent to "trunk" or "nightly". We deliver straight to a testing environment for "develop" plus one instance for every release branch we're maintaining. Testing is next up on the list, at both a "testers bill time to write selenium tests" and "developers roll unit tests into their time from now on" level. Paul MaudDib fucked around with this message at 06:25 on Jun 4, 2015 |
|
#
¿
Jun 4, 2015 05:54
|
|
)
|
Ithaqua posted:The other big challenge is to get people to start using feature flags and short-lived dev branches so you can ship your code even if a feature is half-completed. The killer is usually database stuff -- it's hard to get into the mindset of never (rarely) introducing breaking database schema changes. There are also dedicated schema management tools which attempt to address this. I haven't used one, but one that caught my eye was Flyway. The gist is that you dump some baseline content set or schema as "REV 001" and then store all schema/content changes (it's important that it be all changes) as additional revisions. By walking through the set of revisions you end up with whatever state the program expects. This can be done as part of a Maven build script or whatever. I guess I use a ghetto version of that. What I have is a set of scripts that produce backups, which I check into git. You can then run the scripts and instantiate a database. In order for this to work well the ordering of the rows needs to be stable (Postgres doesn't guarantee any ordering of exported data), and you want one SQL file per table. I hacked up a version of pg_dump_splitsort. It also doesn't work well during add/removing columns since that means every row changes - typically I make one commit for the schema change and then a second commit for any content changes so that the changes stand out. You can obviously use branches, etc if you want to have multiple incompatible versions under development concurrently. Flyway would be nicer there because you see the exact change that's occurring rather than a gigantic block of changed rows. We do more content changes than schema changes so it works OK in practice. Lord Of Texas posted:"Breaking" database schema changes can be part of a toggles/feature flags approach too, the key to making that easy is having an SOA architecture where you don't have 50 different apps reading and writing from the same database tables. As sort of a half-way approach, I broke our database into "content" and "logging" services. We have a few really legacy versions still kicking around and there's no money to rewrite and improve them. To let us move forward I broke it into "content" and "logging" databases - all the instances share the logging DB for configuration, user data, and logging, but each one can have its own version of our content set. It's not perfect but it's at least unjammed our forward progress. The really stupid thing is that the most legacy version of all is a JSON implementation which a phone app talks to, so we could make the web front-end be a wrapper around the JSON service and roll our improvements back into the phone app... Paul MaudDib fucked around with this message at 00:39 on Jul 15, 2015 |
|
#
¿
Jul 15, 2015 00:34
|
|
|
So previously I've been running config as a "sidecar" folder - in git/myproject/ we have two directories of interest, "maven-dir" and "config-dir". Config-dir contains many different files, most of which fall into the category of either credentials (eg SQL server target and user/pass, or external service credentials) or content (eg some PDFs that we link to). Previously, each "build configuration" would have its own config subfolder - f.ex we have "config-dir/develop", "config-dir/master", "config-dir/testing", etc. Deployment involved manually copying config files to target, which isn't a trivial process since we can't directly access production. Each branch would have only its own appropriate config files - eg "master" would only have "config-dir/master". That's not my ideal system, it's just codifying the state of things before I joined the project. So for a revamp, how can I make this better? So far I've already added Maven build-profile-based control of config-dir lookup. So when you're running branch "master" there's a property ${myproject.configdir} that specifies "/config-dir/myproject-master". You still have to copy it manually for now. In terms of config - there are some small differences between dev/testing and production environments (eg different database servers, different external service credentials). Right now I think it's probably manageable with Maven build properties since there's only like a half dozen different fields, tops. We have property files in the resources folder which control this, and it's already set up for property injection, for now I could just do that. Let's say there were more than a half dozen different config fields that varied - what would be the management strategy there? Do all the potential variations of the credential files ride along, and I just specify which one of them I want for a particular build? Or do I want a credential management type service? In terms of content - I think all differences occur across version numbers. We do have some versions that are super legacy and can't be touched. The strategy I've implemented for emergency bug control is that we can run multiple instances of the app at different versions, and we use Apache to map them back to specific older versions if necessary. In other words we can use mod_proxy to remap the "master" build's URL "/myproject/" in the case where URL is "/myproject/client3/" to "/myproject-v2.1.0/client3/" to hold them back to a specific version. This isn't ideal, but the previous strategy of "a different instance for every client" wasn't going to be sustainable. So we do need to maintain the capability for multiple independent instances, and they need to not clobber each other on deployment. Right now, to be honest we could just build all the content right into the WAR file. It's only like 10MB of files tops, so our app goes from 15MB to 25MB. No big deal. Let's say it was significantly higher - how should we handle deploying multiple instances of the content if it amounted to 1gb for each instance? Maybe we have version-numbered directories or something, so for "content/pdf1.pdf" you go to "/config_dir/myproject-v2.1.0/content/pdf1.pdf"? And then you "git archive" or rsync it across from a management server? I feel like I'm getting my head around the rest of the CI/CD concepts but the whole configuration management thing is a mystery to me. Paul MaudDib fucked around with this message at 02:48 on Jul 17, 2015 |
|
#
¿
Jul 17, 2015 02:42
|
|
|
Ithaqua posted:What type of application is this? I assume from hearing "Unity" that it's a desktop application. Continuous integration is easy: You build it. You run code analysis. You run unit tests. However, you can't really do "continuous" delivery of desktop applications, except maybe to QA lab environments for running a suite of system/UI tests. When you're dealing with desktop applications, the best you can do is publish an installer or something like a ClickOnce (ugh) package. Well, there's technically no reason you couldn't push builds/patches directly to clients with a Blizzard-style downloader daemon. You kinda mentioned this as the "QA lab environment" but I'd just like to point out that you can definitely do automated integration tests on desktop apps if you want. If Riot can run automated integration tests on League of Legends then you can run them on pretty much whatever, if you have the willpower. Probably much tougher to design in after the fact, of course. As always. I'm curious what tooling Microsoft has for this, though. Please elaborate. Boz0r posted:What are peoples' thoughts on TeamCity vs Jenkins? I've never used TeamCity but I've used Jenkins and Bamboo and I thought Jenkins was steaming garbage in comparison to Bamboo. Any given task takes probably 5 times as long to configure. On the other hand once you configure them both of them Just Work. Doing a one-time configuration almost certainly will not be a major component of the project's time. I did like that Jenkins has unlimited agents but 2 local agents isn't a huge restriction for small-project use-cases.  Ithaqua posted:That's why the deployment piece is being foisted off onto configuration management systems for the most part. Overextending a build system to do deployments sucks. Plus builds pushing bits encourages building per environment instead of promoting changes from one environment to the next. You can certainly automate building a container image and trigger some sort of push mechanism but yeah, definitely don't write deployments in CI tools. Generally speaking you can automate pretty much anything that can be defined as an artifact. You can version/build databases using Flyway, or build container images for Docker, or an installer for a desktop app, or whatever. Then it's perfectly easy to push them to a repository (if that makes sense for the artifact type). Making the server itself an artifact is yucky though. sunaurus posted:Thanks, that sounds great, but from what I can tell, it only runs on windows? Sadly, I don't have any windows boxes. On top of the other ways to get ssh on Windows I'm going to plug Cygwin. Yeah POSIX-on-Windows is an ugly hack but for anything in its package library it works great and you can actually build a surprising amount of the rest pretty easily. It's super nice to have a POSIX environment for Python/Perl, as well as a complete suite of standard Unix-style command-line tools. For a Windows-native client PuTTY is the standard recommendation but it actually has some annoying downsides that a real openssh client on Cygwin doesn't. And on top of that it's hosted on an HTTP-only website which is kinda derpy for a security-critical tool. Paul MaudDib fucked around with this message at 23:26 on Sep 22, 2016 |
|
#
¿
Sep 22, 2016 23:20
|
|
|
revmoo posted:I've used Jenkins a few times and finally this time around fallen in love with it. It takes a lot more hacking and configuration, but once you have it running it's awesome. I think the best thing about it is that its popularity means there is a plugin for everything. On that note, Maven isn't restricted to just building Java. There is a "pom" target that will let you run a generic artifact build, and that lets you leverage Maven's enormous plugin library. You can build everything from a database schema (Flyway plugin) to a Docker container, and just use Jenkins to manage "build scripts".
|
|
#
¿
Sep 23, 2016 22:59
|
|
|
I'm trying to set up a simple docker setup to learn (container my home fileserver's services). Just to be sure I'm understanding this right: you create a data volume and that's an overlay on the base state of the container that's persisted between runs of the app container? And you can then update the app container independently for point releases without nuking your data? What's the advantage of calling it a "data volume" vs just mounting a host dir in and calling it a day? More ability to scale when you move to a swarm/compose-type system? With a swarm/compose/whatever system, how do you handle the failure of something stateful like a database server where "spin up another" isn't an option? Replication across multiple nodes? One master instance? (Postgres's ability to unfuck itself from the WAL should be handy here) Is there a way for container apps to programmatically signal that they think a service is offline and vote for a reboot of a dead service container in a swarm-type system? For swarm-type setups, what is the best way to accomplish logging for the swarm in a centralized application for debugging/health monitoring? Paul MaudDib fucked around with this message at 02:54 on Jun 8, 2017 |
|
#
¿
Jun 8, 2017 02:34
|
|
|
NihilCredo posted:Surely it's fine to dockerize the database application, though? If you keep your data folder outside of the container, then the docker container running doesn't hold any long-term state, and stopping the container is equivalent to SIGKILLing a regular database process (which, at least for Postgres, would be a quite fine and safe thing to do). This seems to be the shortest way from A to B for small-scale setups in my (very novice) opinion. Have a postgres application container which points to a data directory which is mounted in from the host (eg /srv/postgres/). Unless there's pitfalls here with scalability or something? I'm not sure it's a huge practical gain versus just running it as a bare-metal service but at least this way you bring it inside the Docker system and can apply your logging/health monitoring layer or w/e over the top. But I can definitely see an argument that scale-on-demand database instantiation is probably more trouble than it's worth for small-scale applications and you are probably better off just having one DB per docker swarm or whatever. I'll take a look at Kubernetes too.
|
|
#
¿
Jun 8, 2017 17:52
|
|
|
|
| # ¿ Apr 29, 2024 02:32 |
|
|
Vulture Culture posted:Also awesome for ephemeral test environments for developers. Static infrastructure is fine for prod and maybe staging, but gently caress it in the face everywhere else. I've been pushing heavily for this at work, making a little headway. I'm trying to untangle the classic knotted-up legacy Java webapp at work, we don't have good testing set up yet and there's so many moving parts that I don't know where to start with unit testing. My thought was to "record" HTTP requests and the request/session state each time we call a controller and assert the redirect and relevant state-changes as outputs. So in other words, if you send in a valid create request with parameters X Y Z then there should be an object created in the database with X Y Z, the object created should be attached to the request store as "SelectedRecord", and the request should have been routed to the "success" view. If it failed then you should have an error object attached with a message "field X was not filled". We have a "routing" controller that each request loops through before getting sent to a specific controller, I added a function there that hooks into a DB and dumps the request parameters, request attributes, and session attributes/etc into a PG database as JSON each time we pass through that controller, so we can generate this data on the fly from the Selenium testing we're (slowly) setting up. The advantage of doing it this way is you have a "unit test" for a controller that doesn't require standing up a whole webserver and firing off requests just to do a test (which is very slow), and by starting up an ephemeral test DB instance you should be able to have a bunch of devs working in parallel (or run your tests in parallel for faster build times).
|
|
#
¿
Jun 8, 2017 18:52
|
|