|
I am not a backend person by any means but have a small server-side(ish) personal project I'd like to use as a learning exercise. I think (?) I'd like to use Python. The lazy part of me wants to use Node because I already know Javascript, but I feel like Python is probably the smarter choice. My goal is to accomplish a few relatively simple tasks with this little program: • Monitor a web page for DOM changes • On DOM change: - Send an email, probably using Mandrill, to notify myself of the change - Log the time/date of the change in a database (this is really just to force me to figure out how to connect to and interact with a database) • List times/dates of previous changes on a page, pulling data from the database Does anyone have suggestions re: frameworks or tools in general?
|
#
?
Aug 29, 2016 17:35
|
|
|
|
| # ? May 16, 2024 13:55 |
|
|
kedo posted:My goal is to accomplish a few relatively simple tasks with this little program: To clarify, do you mean: A) you are serving a page and users will be interacting with it and you want to store changes they make? B) You will be pulling a remote page and comparing it to the last time you fetched it to see if things changed?
|
|
#
?
Aug 29, 2016 17:45
|
|
|
Lumpy posted:To clarify, do you mean: A) you are serving a page and users will be interacting with it and you want to store changes they make? B) You will be pulling a remote page and comparing it to the last time you fetched it to see if things changed? B
|
|
#
?
Aug 29, 2016 17:54
|
|
|
Do you really care about the DOM itself changing? Can you just trigger if there is any file diff at all?
|
|
#
?
Aug 29, 2016 18:02
|
|
|
Skandranon posted:Do you really care about the DOM itself changing? Can you just trigger if there is any file diff at all? Any file diff would be fine as an initial solution, but ultimately it'd be nice if I could limit it to a specific set of elements in the DOM. Imagine the page in question has a list of times when a meeting room has openings, and it's always booked. I want to be able to see when a time slot opens, but I don't need to be notified if, say, the developer changes the ?v=# on a CSS file.
|
|
#
?
Aug 29, 2016 18:11
|
|
|
Dumb question, but is this meeting tracking site something with a public API? Something you could tap into with a script and a published access method rather than screen-scraping? (We're focusing on this because the "monitor a remote site for DOM changes" part is the trickiest/most novel thing in your use case)
|
|
#
?
Aug 29, 2016 18:20
|
|
|
Data Graham posted:Dumb question, but is this meeting tracking site something with a public API? Something you could tap into with a script and a published access method rather than screen-scraping? Nope. The meeting tracking is just an example. The real page in question is the result of some weird proprietary time slot tracking system, and it doesn't have a way to publicly access the underlying data except through the website. If it's a weird use case we can just assume a file diff would work for the time being.
|
|
#
?
Aug 29, 2016 20:59
|
|
|
kedo posted:I am not a backend person by any means but have a small server-side(ish) personal project I'd like to use as a learning exercise. I think (?) I'd like to use Python. The lazy part of me wants to use Node because I already know Javascript, but I feel like Python is probably the smarter choice. I'd separate the tasks. First you want to get and save the info: With Node you can achieve this using Cheerio + Request. Request for downloading the page, Cheerio for parsing it and getting the data you want to compare. It has been years since the last time I used Python but I remember Scrapy did everything, from downloading to parsing. Once you have the data save it into a database. Then, using whatever web framework (Express, Django, Flask, etc) you want to learn to use pull the info from the database and display it. lunar detritus fucked around with this message at 23:02 on Aug 29, 2016 |
|
#
?
Aug 29, 2016 22:58
|
|
|
kedo posted:Nope. The meeting tracking is just an example. The real page in question is the result of some weird proprietary time slot tracking system, and it doesn't have a way to publicly access the underlying data except through the website. Either way, I think you are fine using JavaScript, NodeJS allows for doing pretty much everything Python could do. I would definitely focus on the file-diff solution first, as you can probably complete your full stack with that. If you get hung up on DOM tree parsing and comparing, you may never finish.
|
|
#
?
Aug 30, 2016 00:48
|
|
|
Could anyone recommend a REST best-practices guide that they would consider authoritative/insightful? I know the general principles and I'm looking for any extra guidance on implementing resources in a basic RESTful CRUD app.
|
|
#
?
Aug 30, 2016 00:54
|
|
|
If I have IntelliJ Ultimate Edition do I need PyCharm Professional? I currently have licenses for both but it seems like with all the plugins installed in IntelliJ, there's no need for PyCharm? I'm confused...
|
|
#
?
Aug 30, 2016 01:01
|
|
|
Paul MaudDib posted:Could anyone recommend a REST best-practices guide that they would consider authoritative/insightful? I know the general principles and I'm looking for any extra guidance on implementing resources in a basic RESTful CRUD app. Despite the source, this is actually a decent guide. https://github.com/WhiteHouse/api-standards
|
|
#
?
Aug 30, 2016 01:35
|
|
|
kedo posted:I am not a backend person by any means but have a small server-side(ish) personal project I'd like to use as a learning exercise. I think (?) I'd like to use Python. The lazy part of me wants to use Node because I already know Javascript, but I feel like Python is probably the smarter choice. a great Python library for extracting data from HTML is "beautiful soup"
|
|
#
?
Aug 30, 2016 03:37
|
|
|
Does anyone use Google Apps for email for clients? I want to set up a client with email on Google Apps. I already have an Apps account so the ideal way to do it would be to just add their domain to my account and set them up as a user under my organisation. But is that breaking the terms? Since really they're a whole separate organisation? If I end up with a bunch of different domains and users managed from my single account would Google get pissed? The alternative would be to set up a brand new Apps account for the client, with just a single user. But then I'd only be able to login and manage things using their username and password, which isn't ideal... e: looks like there is something like a Google Apps reseller account which lets you administer multiple organisations from a single user account, but you need loads of users / certifications to qualify. Dunno why they set the bar so high... fuf fucked around with this message at 12:53 on Aug 30, 2016 |
|
#
?
Aug 30, 2016 12:39
|
|
|
Thanks folks, all your input is much appreciated!fuf posted:Does anyone use Google Apps for email for clients? A better question might be, "Do I really want to manage a client's email for them?" Unless you're getting paid a lot, it's probably not worth the hassle. Email is a contentious subject and can require a lot of maintenance. Otherwise, if you really do want to do it, have them set up the account and give you the password imo. Better to let them technically own their email rather than you owning it.
|
|
#
?
Aug 30, 2016 12:53
|
|
|
kedo posted:A better question might be, "Do I really want to manage a client's email for them?" Unless you're getting paid a lot, it's probably not worth the hassle. Email is a contentious subject and can require a lot of maintenance. I would love not to manage any email but unfortunately it's one of the curses of being a "hey let me take care of your whole web presence"-style freelancer. But yeah I charge them a lot. kedo posted:Otherwise, if you really do want to do it, have them set up the account and give you the password imo. Better to let them technically own their email rather than you owning it. This makes sense and would maybe work, but ideally I would do the whole setup for them and then invoice them seperately... Maybe it's just not feasible.
|
|
#
?
Aug 30, 2016 13:09
|
|
|
I don't know the exact workings but a few of the local IT shops around here do something similar with Office365. It be worth looking into.
|
|
#
?
Aug 30, 2016 15:56
|
|
|
fuf posted:This makes sense and would maybe work, but ideally I would do the whole setup for them and then invoice them seperately... Maybe it's just not feasible. I mean, you could do that. There's nothing stopping you really. I've just found it creates unnecessary headaches when one of you inevitably fires the other or you decide it's no longer worth your time to manage their email. Maybe set it up with a throwaway Google account and then just turn that account over to them if you need to in the future? e: also fuf posted:I would love not to manage any email but unfortunately it's one of the curses of being a "hey let me take care of your whole web presence"-style freelancer. But yeah I charge them a lot. I feel you, but jesus email is just not worth it imo. It's one thing to be the person whose neck is on the line if "the website is down!" but it's another thing entirely when it's 1:00 AM and "I can't find the email Steve sent me three months ago with the numbers for our call with our client in India and you are LOSING US BUSINESS!" It is one step too far into IT-land, which is a place I hope to never visit again.  PTSD flashbacks all up in this post.
kedo fucked around with this message at 16:20 on Aug 30, 2016 |
|
#
?
Aug 30, 2016 16:15
|
|
|
kedo posted:I feel you, but jesus email is just not worth it imo. It's one thing to be the person whose neck is on the line if "the website is down!" but it's another thing entirely when it's 1:00 AM and "I can't find the email Steve sent me three months ago with the numbers for our call with our client in India and you are LOSING US BUSINESS!" It is one step too far into IT-land, which is a place I hope to never visit again. Man I agree completely, but I dunno what I can do about it, at least at the level I'm currently at. For most of my clients I do everything: register the domains, build the sites, host the sites, update the sites, set up their email. They have zero clue about anything except that they better pay the invoices I send them on time. I'm talking the kind of clients where you have to patiently explain over and over again that a domain name is not the same thing as a hosting package. They want a single "web guy", so if I turn round and say "oh I don't do email" (which I would love to do) then they a) won't understand, and b) will find someone else. I try and draw the line at IT support: I send them the email account credentials, but I don't explain how to set it up on their iphones or whatever. They still ask me though, because they don't understand the difference between local IT support and web support (why should they?). That's why I'd really like to move everything over to Google or maybe Microsoft (thanks Calidus), then I can just say "here's your username and password, tap the big google logo" instead of talking about incoming and outgoing mail servers and port numbers and poo poo.
|
|
#
?
Aug 30, 2016 17:42
|
|
|
Google Apps for Business is pretty good for email, I've used it/set it up for probably 4 different clients. What's nice is you can end run into Google Drive/Docs and avoid DropBox for file sharing/local file servers as well. Google Calendar is a bit harder to get everyone adopted on, but I believe it works better for iPhones now than it did a couple years ago last time I tried. We used to use Google Talk quite a lot internally but the move to Hangouts basically killed that, and since then I've been mostly suggesting Slack. In all these cases I don't manage any emails (I am apparently a higher class of freelancer than fuf  ), the Google interface is simple and powerful enough that once whoever is "in charge" actually gets in there a little lightbulb goes off and they want to manage it themselves. That might be from working mostly in startups though.
|
|
#
?
Aug 30, 2016 17:54
|
|
|
Yeah I get it. Agreed with Saramouche, if you're going to set clients up with email it may as well be Google Apps.
|
|
#
?
Aug 30, 2016 17:56
|
|
|
Scaramouche posted:(I am apparently a higher class of freelancer than fuf The bar has been set pretty low  To be fair I kinda target the "computers are scary, I don't wanna deal with it" demographic on purpose because my only asset as a developer is that I write clear, friendly emails.
|
|
#
?
Aug 30, 2016 18:47
|
|
|
fuf posted:The bar has been set pretty low This is smart. I approve.
|
|
#
?
Aug 30, 2016 19:19
|
|
|
If you can write a coherent email you are objectively better than 99% of developers. I don't even give a poo poo how your code looks.
|
|
#
?
Aug 30, 2016 20:20
|
|
|
too me, agreed I am my friend blessings of god be with you
|
|
#
?
Aug 30, 2016 21:40
|
|
|
Scaramouche posted:too me, agreed I am my friend Stop hacking into my oDesk inbox.
|
|
#
?
Aug 30, 2016 22:10
|
|
|
kedo posted:If you can write a coherent email you are objectively better than 99% of developers. I don't even give a poo poo how your code looks. Yeeeep. That said, I think offering full IT support is no fun at all. These days fastmail.com is our go to for providing email, mailgun.com for transactional delivery, so my work only focussed on website specific stuff.
|
|
#
?
Aug 30, 2016 22:47
|
|
|
kedo posted:If you can write a coherent email you are objectively better than 99% of developers. I don't even give a poo poo how your code looks. The understanding email is an old problem that has been fixed in the interim. Please maintain the emails that is still of relevance to the correct understand. *ticket closed*
|
|
#
?
Aug 31, 2016 08:00
|
|
|
Help. I am bleeding from the ears. I am doing a stupid 'counter' thing wherein if someone submits a form, they go to a 'thanks' page. When the thanks page loads, it increments a text file (civcount.txt) by means of including a php include (civcount.php). Then the first page has a message that says 'join <include civcount.txt> other signers'. All four of these files are in the same directory. In 'thanks': php:<? <p><b>Join <?php include('civcount.txt'); ?> other signees!</p> // this loving works fine, shows the number <p><?php include('civcount.php'); ?></p> // this wont loving work, just blank and doesn't increment ?> php:<?php $filename = 'civcount.txt'; // our counter file $fp = fopen( $filename,"r"); // open it for READING ("r") $counter = fread($fp, filesize($filename) ); // read in value fclose( $fp ); // close it whilst we work ++$counter; // increase the counter by one print "$counter"; // print out the new value $fp = fopen( $filename,"w"); // open it for WRITING ("w") fwrite( $fp, $counter); // write in the new value fclose( $fp ); // close it ?> In closing, what the everloving gently caress? God knows I can use a database but this is on a 50 year old drupal 6 site and damnit to hell. PS: I've tried adding the full URL, the full system path, the real_path and every combination I can imagine. Examples, suggestions welcome. PPS: php:<?php ini_set( 'error_reporting', E_ALL ); ini_set( 'display_errors', true ); include('civcount.php'); ?> pipebomb fucked around with this message at 22:23 on Aug 31, 2016 |
|
#
?
Aug 31, 2016 21:54
|
|
|
Won't that just result in file locks anyway? e.g. 2 people do it at once, which one gets incremented?
|
|
#
?
Aug 31, 2016 23:03
|
|
|
^^^^ Even database locks if you have enough writes per second  pipebomb posted:I am doing a stupid 'counter' thing wherein if someone submits a form, they go to a 'thanks' page. When the thanks page loads, it increments a text file (civcount.txt) by means of including a php include (civcount.php). Then the first page has a message that says 'join <include civcount.txt> other signers'. Your code won't be executed if you're not logged in and the page output is cached on the 'thanks' page (if it's managed by drupal).
|
|
#
?
Aug 31, 2016 23:25
|
|
|
^^ I'm cheating by using include files in the nodes. I know. I am an rear end. ^^ I somehow managed to make this work - it was an issue with the paths after all. I'll test multiple users at the sae time tomorrow to see if I have any lockups. php:<?php getcwd() . "\n"; chdir('/full/loving/path/on/the/server/then-the-subfolder/'); include('civcount.php'); getcwd() . "\n"; chdir('//full/loving/path/on/the/server/'); //note lack of subfolder ?>
|
|
#
?
Aug 31, 2016 23:44
|
|
|
Jeez, that's about the most complicated way I can think to do that, can't you just use variable_set()/variable_get() to persist the count in the database? And yeah without any file locks if two users do submit at the exact same time you'll probably get some funky count stored in the text file.
|
|
#
?
Sep 1, 2016 09:25
|
|
|
Alright, another Drupal question. I'm looking for a module that's more or less equivalent to Advanced Custom Fields for WordPress. I've been playing with Paragraphs which seems to be perfect, except it is buggy as all hell in Drupal 8. Buggy as in "data is lost on post" buggy. Are there any other options out there? I'd use vanilla fields, except I need to give my client a number of field pre-set field groups they can choose from to combine in different orders to create different page layouts, sort of like a flexible content block from ACF. I tried a module called Bricks but I can't even install the thing without seeing a bunch of errors. They're supposedly trying to do a merge with Paragraphs anyways, so that seems like a nonstarter. e: Also, honest opinions on Drupal 8? My client really got the hard sell from the vendor we're working with, and now that I'm actively trying to get their site set up I'm running into tons of problems finding modules that A) have been ported to 8, and B) actually work. I know it's been out for awhile, but it seems to have very, very poor module support at the moment and I don't have the budget to fix buggy modules/code new modules from scratch. e2: Never mind I guess, I found a workaround. Bleh. Maybe I'm just not ub3r developer enough to appreciate it, but Drupal makes me very unhappy and I do not enjoy working with it at all. kedo fucked around with this message at 01:07 on Sep 2, 2016 |
|
#
?
Sep 1, 2016 22:37
|
|
|
What are some good examples of API documentation done right? I remember the Stripe one was often pointed to as a good example. I remember seeing some interactive API docs where you could actually run requests, can't think of any off hand tho
|
|
#
?
Sep 1, 2016 23:50
|
|
|
Swagger (or OData which can generate Swagger). 1: Documentation. 2: Online testing of the API. 3: Automated generation of platform SDKs from an API decl. 4: Automated generation of API decl from service implementation. 3 & 4 are highly a work in progress and are completely poo poo for some combinations.
|
|
#
?
Sep 2, 2016 03:43
|
|
|
I've been tasked with creating a Chrome extension that takes the shape of a side-toolbar injected on certain pages. It is doing what we wanted it to do, but now it is the time to style it and I'm running into some issues regarding some of the pages' agressive styling, namely that it is also affecting the elements of my toolbar. I've read up on CSS resets and managed to apply it only to my elements, but it was a bit too aggressive in that every element reverted to pretty much text only. This would be fine, but most things CSS that are beyond "slap boostrap on the page; add classes" are beyond me. I also tried namespacing the bootstrap css using an automated prefixer site and besides the resulting css being horrible, even with the namespaced sheet the toolbar differed amongst the sites it should work on. So, I don't know what else to try. I imagined slapping the bootstrap CSS would have worked, because it has its own reset rules. But apparently badly-made CSS can overcome all selectors.
|
|
#
?
Sep 5, 2016 14:37
|
|
|
EDIT: I can't read.
|
|
#
?
Sep 5, 2016 17:32
|
|
|
HardDisk posted:I've been tasked with creating a Chrome extension that takes the shape of a side-toolbar injected on certain pages. It is doing what we wanted it to do, but now it is the time to style it and I'm running into some issues regarding some of the pages' agressive styling, namely that it is also affecting the elements of my toolbar. I'd add an id to the sidebar, apply something like normalize.css (which should leave you with default styles instead of just text) editing it to prefix everything with the id and then apply your styles also namespacing the classes with the id.
|
|
#
?
Sep 5, 2016 18:06
|
|
|
|
| # ? May 16, 2024 13:55 |
|
|
gmq posted:I'd add an id to the sidebar, apply something like normalize.css (which should leave you with default styles instead of just text) editing it to prefix everything with the id and then apply your styles also namespacing the classes with the id. This and add !important to all you (namespaced) rules.
|
|
#
?
Sep 5, 2016 20:55
|
|