|
nielsm posted:If your whole data set is "small" enough that it can fit in maybe a few MB of JSON data, implementing everything in client-side JavaScript would be very reasonable. Then you can just have a static HTML file with some static JavaScript, loading the dataset as a static JSON file. baka kaba posted:I don't really know much about web dev, but from my experience of learning a bit about this stuff, you're looking at it the wrong way. If you want to make a modern web page, you don't really learn JavaScript and then code up the scripting and html in notepad, from the ground up. People used to do that, when websites were simpler, but they've built tools that let you put together much more advanced stuff The web dev landscape is super weird, especially to someone from academia who's used to tools doing 1 thing really well and stringing them together in a bash script. I wanted the shortest path to creating this idea and thought frameworks were the long way but apparently they're like a bash script/java lib (say GATK) that you need to figure out how to use. The real issue for me is understanding how front end interacts with backend but I can simplify this by pushing the computing onto the user/browser. The data isn't sensitive but I might need to use a framework to extend the functionality of the site later on. Dominoes posted:Web dev requires putting together several related skills. It presents you with an intimidating scatter of tutorials, tools and standards. Javascript, and more broadly web-dev, is a wild west of coding. I've found my sweet spot of Typescript/React/Redux/Django-Rest, but you might find success down a different path. Once you've focused more, fire away with specific questions. Can I ask, is this an example what they call a "full stack" (front and back end) set of tools?
|
#
?
May 20, 2018 03:59
|
|
|
|
| # ? May 30, 2024 21:56 |
|
|
Yes, that's basically a full stack.
|
|
#
?
May 20, 2018 06:24
|
|
|
Are there any sort of best practices for taking configuration data from a file? Right now I have a code base that loads properties from a .properties file, but it contains 50+ different variables, some have a limited range of accepted values, and some have an assumed default. At the moment it's very confusing to set up. I was thinking of using a json file instead, but this is all kind of new to me.
|
|
#
?
May 20, 2018 22:47
|
|
|
Use YAML don’t shoot me
|
|
#
?
May 20, 2018 22:55
|
|
|
If it's Java based on the fact you're considering properties files, the HOCON library is pretty good. That's similar to JSON and properties files but immutable and with strong conventions about overrides and other such things.
|
|
#
?
May 20, 2018 22:57
|
|
|
The Fool posted:Use YAML 1337JiveTurkey posted:If it's Java based on the fact you're considering properties files, the HOCON library is pretty good. That's similar to JSON and properties files but immutable and with strong conventions about overrides and other such things.
|
|
#
?
May 20, 2018 23:07
|
|
|
SardonicTyrant posted:Is it hard to set up? I'm hoping to keep this as simple as possible so the next person Assuming that you're using a build script that lets you just drop a dependency in there: http://search.maven.org/#artifactdetails%7Ccom.typesafe%7Cconfig%7C1.3.3%7C Here's the project's main documentation: https://github.com/lightbend/config Assuming that you use the recommended project configuration file name it's just: Config conf = ConfigFactory.load(); and it does all the complicated stuff for you.
|
|
#
?
May 20, 2018 23:12
|
|
|
SardonicTyrant posted:Are there any sort of best practices for taking configuration data from a file? Right now I have a code base that loads properties from a .properties file, but it contains 50+ different variables, some have a limited range of accepted values, and some have an assumed default. At the moment it's very confusing to set up. The best practice is to not take configuration from a data file at all. Modern software development (containers, etc.) have settled on environment variables as a lingua franca for sharing config data into an application. If you can't do that, and there are good reasons why sometimes you can't, something like JSON or YAML might help you. But file formats are file formats; anything works. It sounds like your bigger problem is validation and the like. Since you're using Java .properties files I'm assuming you're using the JVM, and so looking at how Dropwizard pulls in configuration from a YAML or JSON file via object mapping and the use of Hibernate Validator to make sure that you're passing legal values might be helpful.
|
|
#
?
May 21, 2018 05:50
|
|
|
AFashionableHat posted:The best practice is to not take configuration from a data file at all. Modern software development (containers, etc.) have settled on environment variables as a lingua franca for sharing config data into an application. If you can't do that, and there are good reasons why sometimes you can't, something like JSON or YAML might help you. But file formats are file formats; anything works. It sounds like your bigger problem is validation and the like. Since you're using Java .properties files I'm assuming you're using the JVM, and so looking at how Dropwizard pulls in configuration from a YAML or JSON file via object mapping and the use of Hibernate Validator to make sure that you're passing legal values might be helpful. Is there a tutorial for properly using environment variables? *for getting configuration details. The biggest issue is how utterly nonsensical our test cases are designed, but I think I can fix that. SardonicTyrant fucked around with this message at 13:47 on May 21, 2018 |
|
#
?
May 21, 2018 13:45
|
|
|
Suspicious Lump posted:Unfortunately the data presentation I have in mind is a tad more complicated. Essentially it's a clickable heat map. Unfortunately, I think you're right, web dev is drat complicated! Kinda like https://bokeh.pydata.org/en/latest/docs/gallery/image.html or http://bl.ocks.org/ianyfchang/8119685 ?
|
|
#
?
May 21, 2018 14:44
|
|
|
AFashionableHat posted:The best practice is to not take configuration from a data file at all. Modern software development (containers, etc.) have settled on environment variables as a lingua franca for sharing config data into an application. What? Environment variables are a pain in the rear end: they're easy to accidentally leak to other executing contexts, they only really support key=value config (i.e. no complex schema), and they're hard to share/propagate/reuse. Explicit configuration files have none of those problems. Maybe certain domains of modern software development have settled on environment variables because of limiting constraints they need to operate under, but it's not at all clear to me that that means everyone should be using environment variables for their config. Because I work at Google, I lean on protobufs for config: define a proto, make a text file containing the data for that proto, then load the text file into an empty proto instance using your language of choice. So that's the route I'd take. If you don't like protobufs, equivalent setups exist for JSON/XML/etc., I'm sure.
|
|
#
?
May 21, 2018 15:54
|
|
|
I've been messing around with homeassistant and they use the python library Voluptuous for config file validation. I'm not doing any development yet, but its really nice being told exactly what keys or values or whatever I messed up in the config files. In other words, as an end-user I'm really liking good config file handling.
|
|
#
?
May 21, 2018 17:56
|
|
|
TooMuchAbstraction posted:What? Environment variables are a pain in the rear end: they're easy to accidentally leak to other executing contexts, they only really support key=value config (i.e. no complex schema), and they're hard to share/propagate/reuse. Explicit configuration files have none of those problems. Maybe certain domains of modern software development have settled on environment variables because of limiting constraints they need to operate under, but it's not at all clear to me that that means everyone should be using environment variables for their config. If you work at Google, you've got a pretty niche view of the world and the tooling you have available to you in your deploy pipeline probably helps a lot with packaging those protobuf config files into per-environment deployments, yeah? None of k8s, docker-compose, Nomad, etc. really provide that, and there is a rush to the lowest common denominator. Which, TBH, means env vars. I'm an independent software developer and consultant, and a large part of my business is devops/CI/CD stuff (because I have the secret superpower of being able to read documentation and log files, apparently). My post was descriptive, not prescriptive. A lot of it comes down to k8s/docker-compose/etc. being inelegant when used by...well, most people, because most people aren't. Getting configuration data via files into containers sucks--it is doable but it sucks, volumes are an ongoing tire fire--and if you're pulling down open-source/community containers you will overwhelmingly see configuration by environment variables; if you want to not use environment variables, then you are probably building your own from scratch. Given the way that Docker containers are, on the leading not-FAANG edge, becoming--and I'm not saying it's a good idea--de facto standard distribution packages, this is just becoming A Thing. FWIW, while I agree that env vars are often a pain in the rear end, a lot of the downsides, though certainly not all, are mitigated by Our Shiny Container Future. They're a lot harder to leak to other executing contexts and they're usually fed by k8s deployments, docker-compose documents, and the like so they do become more easily shared and reused. (And that they still remain a pain in the rear end for some purposes is exactly why I pointed at Dropwizard/Hibernate Validator.)
|
|
#
?
May 22, 2018 07:13
|
|
|
Having given it some thought, I think I'm going to stick to config files over environment variables. If for no other reason than we don't currently use containers, and I don't want to make our code base more complicated. Also, I think it would require rewriting some Jenkins scripts we have, and I don't understand the scripts enough to risk touching it. Appreciate the help in this thread though.
|
|
#
?
May 22, 2018 12:07
|
|
|
Do what we do, and have config files set environment variables. 
|
|
#
?
May 22, 2018 15:23
|
|
|
AFashionableHat posted:If you work at Google, you've got a pretty niche view of the world and the tooling you have available to you in your deploy pipeline probably helps a lot with packaging those protobuf config files into per-environment deployments, yeah? None of k8s, docker-compose, Nomad, etc. really provide that, and there is a rush to the lowest common denominator. Which, TBH, means env vars. I'll freely admit that I'm not really up on this particular aspect of software development. But are containers really that big a deal? It seems to me that this is mostly focused on highly-scalable setups where you need to be able to just dump more VMs into your website or service by clicking a button. Which is certainly important but I have no idea what proportion of software developers are in that domain. Anyone else want to chime in with their experience or lack thereof on this front?
|
|
#
?
May 22, 2018 16:12
|
|
|
TooMuchAbstraction posted:I'll freely admit that I'm not really up on this particular aspect of software development. But are containers really that big a deal? And while I used to be pretty unhappy with Docker, and parts of it still drive me bugshit, Docker as a whole is...fine, nnow. Some things, like signal handling, are weird, but they're known potholes. More importantly, for a long time it didn't support user namespaces, which presented some security challenges; this is now addressed, and while they're not as secure as VMs I would say that running applications inside of Linux namespaces, through whichever technology you want, is a better call than not doing so. A common pattern is to just use Docker as a packaging mechanism for services and launch a single container on each server in your fleet, and you keep the sharp objects away from your developers (more on that in a second). The bigger question you asked, and where stuff like scaling comes into play, is whether container fleets are/should be that big a deal. And they probably shouldn't, but it seems like everyone and their dog is running k8s these days whether they need it. Depending on where at Google you are, you can probably appreciate what the various fleet technologies offer, but there are obvious questions, mostly unanswered, around how they scale downwards (in terms of people). I think the answer is "poorly," but I've also made a lot of my living the last few years helping companies who decided they wanted to add More Containers To It as an engineering strategy. It's still a leading-edge technology, but my larger customers, including a couple of large financial services companies, are either looking at moving to containers as a deployment strategy or are already doing it. And I think that most of the cloud vendors have decided that this is the way forward, with Google and Azure already having their own k8s-as-a-service offerings and every Amazon homegrown container platform[0] thing being more or less replaced with a k8s service in the not-too-distant future. There will always be people not using them, but I think that as a general trend the die is pretty well cast. I think the overall drive, if unintentional, is to continue the trend of developer specialization. The "modern" environment is a lot of programmers who are either junior or knowledgeable only in a small vertical bashing on parts of a system, with the occasional wizard around to solve the "hard problems" created by everyone else[1]. Containerization tries to facilitate that by reducing the scope of things that developers (think, sometimes mistakenly, that they) need to understand about the software they're working on. Which eventually fails. Abstractions usually do. Our Shiny Container Future gives companies more reasons to pay me when those abstractions fail, so I'm not sad about it, but if it sounds bonkers to you I don't think you're wrong.  [0]: Except Fargate, but Fargate is a money trap for the stupid. Not that running a k8s cluster for a Rails app or whatever is not sometimes a money trap for the stupid, but Fargate is significantly worse. [1]: As an example outside of operations/containers/etc. that's more directly related to writing code--about a year ago I had a client who had tried to implement job backoff in Node by recursively calling a function that returned a Promise, not knowing that doing so has some severe consequences, and called me in when they started seeing their server "crash"--which it wasn't actually doing, it was just in GC hell all the time. A smart group of developers, but uniformly young and ignorant of how the stuff they rely upon is implemented. And not really equipped with the tools to troubleshoot their problems. tracecomplete fucked around with this message at 17:14 on May 22, 2018 |
|
#
?
May 22, 2018 17:12
|
|
|
TooMuchAbstraction posted:I'll freely admit that I'm not really up on this particular aspect of software development. But are containers really that big a deal? It seems to me that this is mostly focused on highly-scalable setups where you need to be able to just dump more VMs into your website or service by clicking a button. Which is certainly important but I have no idea what proportion of software developers are in that domain. Anyone else want to chime in with their experience or lack thereof on this front? No, containers are absolutely a big deal and really they're a smart thing to use no matter the size of your deployment.
|
|
#
?
May 22, 2018 18:01
|
|
|
AFashionableHat posted:I would say so--and for folks for whom they aren't, they will be soon enough. Thermopyle posted:No, containers are absolutely a big deal and really they're a smart thing to use no matter the size of your deployment. Hm, okay. Sounds like I'd better brush up on them then. Thanks y'all.
|
|
#
?
May 23, 2018 05:19
|
|
|
AFashionableHat posted:Getting configuration data via files into containers sucks Why isn't that just a git clone?
|
|
#
?
May 23, 2018 14:45
|
|
|
ultrafilter posted:Why isn't that just a git clone? lol if u commit secrets to git repos
|
|
#
?
May 23, 2018 18:03
|
|
|
ultrafilter posted:Why isn't that just a git clone? This doesn't parse to me. But maybe it's a terminology collision, and a number of frameworks don't help with that. Like, you can credibly call Rails's `routes.rb` "configuration", but that's not an infrastructural definition of configuration. It's just part of your application. When speaking in an infrastructural sense, "configuration" is generally reserved for environment-specific data. This includes secrets, as Pollyanna suggests, but also stuff like "this is my DNS root" and "my Redis server is on this host". Checking those into a git repo doesn't make a lot of sense. And, for reasons that I hope are reasonably obvious, you'd want to run in your production environment the same artifacts (be they JARs, containers, whatever) that you have tested in test/stage/preprod/whatever. They're what you tested--you don't want to rebuild the things and run something untested in production. So, for containers but also for other all-in artifacts like machine images (which for a while were a thing with "immutable servers" or whatever), you need to provide that configuration at runtime. For containers, that basically means environment variables, exposing a file or files (i.e., a JSON config file, but also possibly using /etc/default) via Docker volumes, or a runtime configuration service like Archaius or Consul...but then you have a turtles problem on your hands, in the sense of "where's my Consul cluster?", and you get to do it all over again.
|
|
#
?
May 23, 2018 18:53
|
|
|
No idea where else to put this since the functional programming thread is locked. Trying to teach myself a bit of Elixir and I'm doing the exercises on exercism. Not sure I understand the solution someone posted for the following problem: Problem: quote:Histogram function: Returns a summary of counts by nucleotide. Solution: code:code:
|
|
#
?
May 23, 2018 22:12
|
|
|
I don't know Elixir so this is uninformed chat, but looking around it seems like you're right about the count function needing a function parameter (here's someone else working through the same problem) - so either that person has written their own count function, or they are passing in functions? This output looks suspicious to me: code:I'd try and research this but it's kinda hard to google things like ? and @ 
|
|
#
?
May 24, 2018 00:03
|
|
|
The @ symbol signifies module attributes which can be used as constants. The ? before the character returns its matching codepoint rather than a string representation of said codepoint. I wonder if you're onto something, with the ? essentially being treated as a function itself, but in any case I think the second parameter-function should be returning a boolean rather than an integer, so that doesn't quite line up precisely... Edit: doh I'm an idiot, it's just calling a count that's defined inside that module rather than a library/API. Thanks for pointing me to that! reversefungi fucked around with this message at 00:18 on May 24, 2018 |
|
#
?
May 24, 2018 00:16
|
|
|
Well that makes a lot more sense and provides less horrors, a good ending!
|
|
#
?
May 24, 2018 00:37
|
|
|
I have a table in PowerBI. The table looks like; Report | Group | Reporting Area | Location | Voltage | Title | Reporting Year | Month | Month Year | Target | Actual For each Report Title in the table (34) I have 2018-19 row-by-row with a target for each month. Id like to be able to have a measure for under/over performance against target, and to be able to plot each Report Title across 12 months with target, monthly and ytd performance. I’m having a mare tbh, there seems to be no way of using the initial table bar splitting it apart into a table per report title and then using that? But trying to use a date table is loving up otherwise because it’s be a many to many join?
|
|
#
?
May 25, 2018 16:04
|
|
|
Has yarnpkg ate poo poo for anyone else? I'm seeing it spit out 403s. And this if I try to curl something off of it You've requested a page on a website that is part of the Cloudflare network. The host is configured as a CNAME across accounts on Cloudflare, which is prohibited by security policy.
|
|
#
?
May 25, 2018 23:58
|
|
|
I think it has to do with the rollout of GDPR today.
|
|
#
?
May 26, 2018 02:28
|
|
|
NPM switched to cloudflare, which broken yarn's things that point at npm. It's now back up.
|
|
#
?
May 26, 2018 05:48
|
|
|
Anyone have a good resource for popularity of backend frameworks? All I want to know is frameworks in use by percentage. All the articles I'm finding are for crap like "What frameworks you should learn in 2018!" Historical trends would probably also be useful.
|
|
#
?
May 29, 2018 17:24
|
|
|
There's a site that crawls a bunch of other sites and breaks down what technologies they use and IIRC it has some heuristics to make a best guess at the backend. Unfortunately, I can't recall what this site is called!
|
|
#
?
May 29, 2018 17:29
|
|
|
huhu posted:Anyone have a good resource for popularity of backend frameworks? All I want to know is frameworks in use by percentage. All the articles I'm finding are for crap like "What frameworks you should learn in 2018!" Historical trends would probably also be useful. These days it's all about the SPA and the REST API, except the REST API is being replaced by I hear that Elixir/Phoenix thing is pretty cool though...
|
|
#
?
May 29, 2018 17:39
|
|
|
huhu posted:Anyone have a good resource for popularity of backend frameworks? All I want to know is frameworks in use by percentage. All the articles I'm finding are for crap like "What frameworks you should learn in 2018!" Historical trends would probably also be useful. Stackoverflow Insights has some good info, including their developer survey. But they rely primarily on the usage of their site, and self-reported numbers from developers. https://insights.stackoverflow.com/
|
|
#
?
May 29, 2018 17:55
|
|
|
Oh, here's the one I was thinking about : https://www.wappalyzer.com/technologies There's also: https://builtwith.com/
|
|
#
?
May 29, 2018 18:07
|
|
|
If I want to render SPA DOM code on the server am I stuck with a something running on Node? Can I get any of the big front-end frameworks to hook into my server-rendered DOM + state object?
|
|
#
?
May 29, 2018 19:53
|
|
|
strange posted:If I want to render SPA DOM code on the server am I stuck with a something running on Node? Can I get any of the big front-end frameworks to hook into my server-rendered DOM + state object? On the other hand, if you're transpiling to javascript from a language that has other compilation targets, then the framework you're using may also provide a way to do server side rendering using the other compilation targets so you don't need to use node.js. mystes fucked around with this message at 20:16 on May 29, 2018 |
|
#
?
May 29, 2018 20:09
|
|
|
mystes posted:If you're writing your frontend in javascript, how are you going to render that on the server without using node unless you write another identical version of it in another language? It makes the baby jesus weep, but you can do it.
|
|
#
?
May 30, 2018 00:19
|
|
|
strange posted:If I want to render SPA DOM code on the server am I stuck with a something running on Node? Can I get any of the big front-end frameworks to hook into my server-rendered DOM + state object? i'm almost inclined to say that if you want to render SPA on the server, you could like, consider not making an SPA. SPA is a specific technical term meaning "bloated website that uses a twenty megabyte of js framework to solve the challenging technical problem of making it so if the data in your model changes, your view should also update."
|
|
#
?
May 30, 2018 13:38
|
|
|
|
| # ? May 30, 2024 21:56 |
|
|
strange posted:If I want to render SPA DOM code on the server am I stuck with a something running on Node? Can I get any of the big front-end frameworks to hook into my server-rendered DOM + state object? Theoretically there's no reason you couldn't write your DOM into, say, React js files or Angular template files and serve those to the client. Although you'd have to be careful how you do that so you don't lose the advantages of caching. Node's just the natural use case since the built-in tools use it. Bruegels Fuckbooks posted:i'm almost inclined to say that if you want to render SPA on the server, you could like, consider not making an SPA. SPA is a specific technical term meaning "bloated website that uses a twenty megabyte of js framework to solve the challenging technical problem of making it so if the data in your model changes, your view should also update." Server-side rendering of the first page is a legit technique. Helps with web crawlers and speeding up the initial download and render.
|
|
#
?
May 30, 2018 13:51
|
|
