Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Hammerite
Mar 9, 2007

And you don't remember what I said here, either, but it was pompous and stupid.
Jade Ear Joe

Look Around You posted:

Most modern web frameworks use a Model-View-Controller (MVC) pattern. The three parts are as follows: a "Model", which contains the representation of the data and the rules for manipulating it, a "View" for displaying it to the user, and a "Controller" that decides what needs to be done and how to do it, basically mediating between the Model and the VIew.

So you'll have a Controller get the request (maybe "yoursite.com/blog/1") and decide what to do with it, usually by calling a function that you "route" that request to. That function (still part of the controller) will then handle getting the data from the model, and pass it to the view (which is usually a special template file, typically html with some code in it to do some display logic).

Here is a good beginners tutorial for the Ruby on Rails (which is obviously written in Ruby)

here is an overview and a tutorial for the Django framework, which is written in Python.

Thanks. In your estimation, how much extra hassle would it be to learn django using only shared hosting with no command line access, as opposed to getting a cheap VPS?

edit: Also if I should get out and ask questions in the dedicated language megathreads instead, tell me.

Hammerite fucked around with this message at 12:32 on Apr 10, 2012

Adbot
ADBOT LOVES YOU

tef
May 30, 2004

-> some l-system crap ->

Hammerite posted:

I have seen forums poster tef refer to this as "CGI-style" scripting (or maybe I have misunderstood terms, but that is how I perceived it).

In ye olden days of web development, you put a script in the cgi-bin directory, and when you loaded it in a web browser, the server *ran* the script and returned the output, instead of returning the contents of a file.

You can write cgi scripts in any language, python, ruby and perl all support this. However this turns out to be a little clunky for some things.

Instead, they now load the script into the server and call a function when a request comes in. The scripting language then decides which bit runs what and what gets returned. It's passing more control from the web server to the script.

People have hacked this with php and mod_rewrite - where every url goes through a single index.php file, which then includes all the bits and spits out the output.


The megathreads are excellent places to ask questions about django or rails. Both make good choices to explore other models of web development. There are also places like heroku where you can get your application up and running cheaply.

If I were to recommend anything, it would be django, the documentation is fantastic.

Optimus Prime Ribs
Jul 25, 2007

I didn't see a regular expressions thread, so apologies if this the wrong thread to be posting this in.

I need to parse a log file which contains timestamps and IDs. The ID will only ever be an integer, or a URL, and the log file is always in this format:
code:
00:00:00	[image] 1
00:00:18	http://somesite.com/stuff/foo.html
00:02:43	[image] 3
Currently I am using this expression:
code:
([0-9]{1,2})[:]([0-9]{2})[:]([0-9]{2})[\s]+(\[image\][ ])?([^\n]+)[\n]?
It works, but the problem is that the first line of the log file contains a timestamp of when the file was generated:
code:
1330556493	Wed Feb 29 15:01:33 2012
and the 15:01:33 2012 part is being matched by the regular expression.
Right now my solution is to manually remove that from the log file before parsing it, which is kind of a pain in the rear end.
I can't think of what I can add to the regular expression to ignore that line.

Note:
I can't have that line be omitted; I don't have access to the script which generates the file.

Any ideas?

ToxicFrog
Apr 26, 2008


Hammerite posted:

Thanks. In your estimation, how much extra hassle would it be to learn django using only shared hosting with no command line access, as opposed to getting a cheap VPS?

Depends on the hosting, I'd think. Some stuff comes with a built in "tick this box and we will enable django support for you". If it doesn't I'm not sure if you can set up django without command line access, but it's been quite a while since I set it up so don't quote me on that. (The django megathread would probably know.)

I would like to point out that Django - like most frameworks of this kind - comes with a built in test server. It's not good for deployment, but if you just want to learn you can always just install it locally, run the test server, and connect to it with your browser, no need to deploy to a remote server at all. Unlike PHP you do not need a complete Apache stack to test it.

pokeyman
Nov 26, 2006

That elephant ate my entire platoon.

Optimus Prime Ribs posted:

Currently I am using this expression:
code:
([0-9]{1,2})[:]([0-9]{2})[:]([0-9]{2})[\s]+(\[image\][ ])?([^\n]+)[\n]?

Quick fix: anchor the start of the regex to the start of the line by putting a caret ^ at the start.

code:
^([0-9]{1,2})[:]([0-9]{2})[:]([0-9]{2})[\s]+(\[image\][ ])?([^\n]+)[\n]?
I also rearranged your regex a bit, not sure if it's helpful:

code:
^((?:[0-9]{1,2}:?){3})\s+(\[image\]\s*\d+|\S+)
I'll break it down:
code:
^ # from the start of the line
(
  (?:[0-9]{1,2}:?) # one or two digits, optionally with a colon
{3} # repeated three times
) \s+ # then whitespace
(
  \[image\]\s*\d+ # either the string '[image]', followed by whitespace, followed by a number
  |   # or
  \S+ # a bunch of non-whitespace
)
This has the benefit (?) of your capture groups being, in order, the timestamp and the id/url. (Odds are you can use that commented version verbatim if you have a 'x' or 'ignore pattern whitespace' option in your language.)

edit: Had fixed something in the multiline version but not in the one-liner.

pokeyman fucked around with this message at 19:17 on Apr 10, 2012

Optimus Prime Ribs
Jul 25, 2007

pokeyman posted:

Quick fix: anchor the start of the regex to the start of the line by putting a caret ^ at the start.

Doesn't the caret mean the start of the entire string, not just a line?

pokeyman posted:

I also rearranged your regex a bit, not sure if it's helpful:
code:
^((?:[0-9]{1,2}:?){3})\s+(\[image\]\s\d+|\S+)

This seems to be not working:


I am not super great with regular expressions so I can't spot why, but when I remove the caret this happens:

Carthag Tuek
Oct 15, 2005

Tider skal komme,
tider skal henrulle,
slægt skal følge slægters gang



Unless you configure your regex engine to treat a multiline string as one line, the caret will match the start of each line.

The Gripper
Sep 14, 2004
i am winner
Throwing it into rubular seems to work with no problem, maybe it's regexbuddy being a pain? http://rubular.com/r/60wJbnHnbS

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
I have a potentially dumb question: is there a standard definition of Model-View-Controller in terms of what responsibilities each part has? Every time I see some explaining Model-View-Controller there are always two people who say "No, that's not MVC". Atwood seems to have one definition, but some people feel that the web browser isn't an accurate representation of "MVC", just "separation of presentation and data". Similarly, most MVC frameworks take the "controller" to mean "thing that does URL routes". This doesn't seem to fit the responsiblity of the controller, according to the definition as supplied by Wikipedia (yes, I know it's a terrible source, which is why I'm looking for a better one):

Wikipedia posted:

The Controller receives user input and initiates a response by making calls on model objects. A controller accepts input from the user and instructs the model and a view port to perform actions based on that input.

The Django developers make the claim that Django is not an MVC framework because the C doesn't make sense in the context of the web.

The original paper introducing MVC in the context of Smalltalk is much less clear:

Applications Programming in Smalltalk-80(TM): How to use Model-View-Controller (MVC) posted:

Unlike the model, which may be loosely connected to multiple MVC triads, Each view is associated with a unique controller and vice versa. Instance variables in each maintain this tight coupling. A view's instance variable controller points at its controller, and a controller's instance variable view points at its associated view. And, because both must communicate with their model, each has an instance variable model which points to the model object. So, although the model is limited to sending self changed:, both the view and the controller can send messages directly to each other and to their model.

This doesn't seem to map up with any MVC framework today. I'm asking because, for me, in order for the "MVC pattern" to have any weight, there has to be some common consensus on what an "MVC pattern" consists of. If there is one, and the rest of people are just silly internet argument people, then please let me know.

rolleyes
Nov 16, 2006

Sometimes you have to roll the hard... two?
I think part of the problem is that (as with many design patterns) MVC is an idealised abstraction which doesn't necessarily fit all that well with reality, depending on your application. When it comes to the web, in my experience most people are happy to accept the backend DB as the model. Boundaries start to blur regarding views and controllers since it can be argued that the browser encompasses elements of both - especially when you start throwing AJAX and similar technologies into the mix.

The way I've seen many frameworks (e.g. CodeIgniter - yes I know, ew PHP, etc) deal with it is to have something which amounts to a templating engine as the view and pass all of the grunt work back a level to a controller. On a technical level, when a request comes in the controller receives it and does whatever processing is required (e.g. queries on the model), passes the result (e.g. a dataset) to the view, the view takes that result and does some presentation work on it (e.g. dataset becomes a pretty table) and that is then passed to the browser.

Is it contrived? Yeah, kinda. Again, the pattern doesn't necessarily reflect what is actually possible on a practical level. With a system such as that then at least you keep your presentation logic and your processing/lookup logic separated. Considering the messes which occur when you mix presentation logic with your business logic (for examples of this, see pretty much anybody's first ever PHP website) this can only be a good thing.

rolleyes fucked around with this message at 19:44 on Apr 10, 2012

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
Right. I understand the "M" and the "V"; separation of data from presentation is a good thing. I just don't understand the "C", and I've never been able to find a clear conceptual overview of what it is (or even how many there are) and what its responsibilities consist of.

pokeyman
Nov 26, 2006

That elephant ate my entire platoon.
C talks to M and V, getting data from the model into the view, and getting input from the view into the model.

No Safe Word
Feb 26, 2005

Suspicious Dish posted:

Right. I understand the "M" and the "V"; separation of data from presentation is a good thing. I just don't understand the "C", and I've never been able to find a clear conceptual overview of what it is (or even how many there are) and what its responsibilities consist of.

Models and Views are relatively "dumb" when it comes to "business logic". Models are just concerned about "what is my data structured like and how can people get it?". Views are just concerned about "how do I present the set of stuff I want to present?" The business logic is where your controller comes in. It either contains the business logic or makes the calls to the services/layers that have the business logic. It says hey I need the following data (talking to the Model) so that I can transform it so it may be presented in the View. Similarly when users interact via their View it then determines if it's appropriate to make changes to the data (which it would then tell the Model) about.

nielsm
Jun 1, 2009



Let me join in on the bikeshed talk.

The model stores data in some form.
The view pulls data from the model and presents them.
The controller takes commands for modifying the data in the model.
The view offers the user relevant controller actions.
The controller sends the user to a new view after performing its action.

Extremely simplified in a HTTP context, a view would be what handles a GET, while a controller would be what handles a POST, and the model is not accessed through the HTTP interface.
Of course the way actual frameworks (and web servers in general) tend to be implemented it doesn't make sense to distinguish on view and controllers in the request routing, so they end up looking mostly the same from the application developer's view.

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe

pokeyman posted:

C talks to M and V, getting data from the model into the view

nielsm posted:

The view pulls data from the model and presents them.

There's no consensus, then. Thanks!

Plorkyeran
Mar 22, 2007

To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed
How you divide the responsibilities is relatively unimportant as long as you actually do so and your division isn't completely insane.

pokeyman
Nov 26, 2006

That elephant ate my entire platoon.

Suspicious Dish posted:

There's no consensus, then. Thanks!

Well yeah, if that's all you wanted to know I could've saved you some time :-)

I don't think I've seen two implementations of MVC that were done the same way.

raminasi
Jan 25, 2005

a last drink with no ice
I need to scrape an HTML page. I don't know anything about HTML or XHTML or whether it's well-formed or not but it has this at the top:
code:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML lang=en xmlns="http://ww.w3.org/1999/xhtml" xml:lang="en">
I only need to do this once, on this single file, so creating a robust solution isn't required.

Should I treat this as XML? Or what's the regular expression to parse HTML? :v:

pokeyman
Nov 26, 2006

That elephant ate my entire platoon.
Well it's already invalid XML so I'd say don't try that. Throw the page in a regex tester and dick around until you get what you need. Otherwise you want libxml2.

nielsm
Jun 1, 2009



People have mentioned TagSoup for Python before, for scraping HTML. Haven't tried it myself.

rolleyes
Nov 16, 2006

Sometimes you have to roll the hard... two?

Suspicious Dish posted:

There's no consensus, then. Thanks!

Pretty much. I tend to view it (hurrr :haw:) in the same way as pokeyman with the controller acting as a middleman, but really whatever works for you (or whatever is dictated to you by the framework you're using) is fine. Like Plorkyeran says, the important thing is to attempt to separate your data from your logic from your presentation and to do so in some sort of sensible way.

If it helps I tend to think of it in a similar way to building a set of related classes in an OOP language - they have interdependencies but insofar as possible they should not need to know the inner workings of each other. Your controller shouldn't care how the model retrieves data when asked to, or how the view presents the data given to it. The model shouldn't really care about the view at all, and vice versa.

The general idea is that (assuming the data doesn't change) you could rip out your view and replace it with a new one with minimal changes to the controller and no changes to the model. Similarly you ought to be able to rip out the model and replace it with a new one with minimal changes to the controller and no changes to the view. It's all about encapsulating functionality in the appropriate place.

rolleyes fucked around with this message at 23:11 on Apr 10, 2012

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
lxml.html (libxml2's HTML parser) works on invalid XML, and was recently retrofitted to use the HTML5 parsing algorithm.

EDIT: whoops, thought this was the Python thread. libxml2 has an HTML parser that works fine.

Suspicious Dish fucked around with this message at 22:18 on Apr 10, 2012

ComptimusPrime
Jan 23, 2006
Computer Transformer

pokeyman posted:

Well yeah, if that's all you wanted to know I could've saved you some time :-)

I don't think I've seen two implementations of MVC that were done the same way.

Most MVC architecture seems to me like Layered architecture where each layer is named "Model", "View", and "Controller".

BigRedDot
Mar 6, 2008

I think the confusion comes because people often start out with an "MV" architecture, basically just hooking a view up directly to a model. But if your application gets more complex, one or the other of those pieces has to get more complex. If you want to support different kinds of views, for example, and you've put lots of logic in your view, are you going to repeat all that logic, more or less, in each view? Some people prefer to keep the models and views very simple, and factor out more complicated logic to a thing called a controller, that mediates between models and views.

The Born Approx.
Oct 30, 2011
Is there a thread for parallel programming questions? Specifically I am having issues running OpenMPI executables on a quad core machine, where it doesnt seem to acknowledge that I have multiple slots on the CPU. Any OpenMPI users around here?

TheMover
Jun 24, 2010
I bought a thing that is physical and has wires that go into the computer box. I want to talk to it but I speak more civilised languages whereas it only understands assembler, is there any neat resources (books preferably but if there's a kickass website, pdf or video out there I'm happy to hear it) for learning this language? If there's even anything more specialised-but-useful out there, it's a PIC microcontroller to be specific.

I'm basically looking for recommendations before I go and splash some cash on something lovely.

TheMover fucked around with this message at 22:23 on Apr 11, 2012

Blotto Skorzany
Nov 7, 2008

He's a PSoC, loose and runnin'
came the whisper from each lip
And he's here to do some business with
the bad ADC on his chip
bad ADC on his chiiiiip
PIC asm is kind of gnarly but workable; this being said, there are two C compilers for PIC in common use that would make your life much easier (one by Microchip and the other by CCS, if I recall correctly).

Gilg
Oct 10, 2002

I'm writing a web API (GET /stuff, GET /stuff/1) and some of the fields to be returned are expensive to compute. I'd like to make it so that getting the collection will not return those fields, but getting an individual one will. Is that a good approach? If so, should I have both calls' JSON schema match and have nulls in the collection's version where the expensive fields are, or just have different JSON schema?

maskenfreiheit
Dec 30, 2004
.

maskenfreiheit fucked around with this message at 21:28 on Apr 28, 2019

Plorkyeran
Mar 22, 2007

To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed
find recurs into subdirectories by default, so find . -size -13M -exec rm {} \; should be all you need to delete all files under 13 MB in the current directory and all subdirectories.

Strong Sauce
Jul 2, 2003

You know I am not really your father.





Gilg posted:

I'm writing a web API (GET /stuff, GET /stuff/1) and some of the fields to be returned are expensive to compute. I'd like to make it so that getting the collection will not return those fields, but getting an individual one will. Is that a good approach? If so, should I have both calls' JSON schema match and have nulls in the collection's version where the expensive fields are, or just have different JSON schema?

Either this is really vague or you are just using the wrong terms because I don't understand what you're trying to do at all. A field is too expensive to compute? Like a field in a row of a database? Or are you talking about rows?

Gilg
Oct 10, 2002

Strong Sauce posted:

Either this is really vague or you are just using the wrong terms because I don't understand what you're trying to do at all. A field is too expensive to compute? Like a field in a row of a database? Or are you talking about rows?

Sorry, I'll try to be more clear.

I've got a web API that returns JSON like so
code:
GET /stuff
[{"id": 1, "name": "blah", "cost": <something that requires a lot of joins>},..]
code:
GET /stuff/1
{"id": 1, "name": "blah", "cost": <something that requires a lot of joins>}
The "cost" field, when called for the entire stuff collection, would take too long to return in a reasonable amount of time, but for a single entry isn't too bad. I'm trying to figure out if it's a better practice to keep the JSON schema for the two queries the same and have cost be null for the collection query or to remove the cost field entirely from the collection JSON or if neither is a good idea and I need to figure something else out.

Edit: That "something else" may be to redesign the tables so cost can be included in the collection query and have it perform reasonably well, but that will be quite difficult. I understand, though, that's probably the best overall solution.

Gilg fucked around with this message at 00:50 on Apr 12, 2012

Carthag Tuek
Oct 15, 2005

Tider skal komme,
tider skal henrulle,
slægt skal følge slægters gang



If you want to get rid of now empty subfolders afterwards, this is pretty handy:

find -depth -type d -empty -exec rmdir {} \;

nielsm
Jun 1, 2009



Gilg posted:

The "cost" field, when called for the entire stuff collection, would take too long to return in a reasonable amount of time, but for a single entry isn't too bad. I'm trying to figure out if it's a better practice to keep the JSON schema for the two queries the same and have cost be null for the collection query or to remove the cost field entirely from the collection JSON or if neither is a good idea and I need to figure something else out.

I wouldn't recommend including it with a "null" value, as that could be interpreted (by folks who don't reads your docs) as "no value for cost will ever exist for this item".
Exclude the field entirely from the returned JSON and document that it is only returned in particular circumstances.

ComptimusPrime
Jan 23, 2006
Computer Transformer
Is there any way you can separate the call to retrieve the price? So after you serialize the JSON for whatever item it is you are selling, you can just have whoever is using it request the prices for those items that they need?

Can you maybe give a bit of information about why calculating the cost requires so many joins in the database?

Strong Sauce
Jul 2, 2003

You know I am not really your father.





Gilg posted:

Sorry, I'll try to be more clear.

I've got a web API that returns JSON like so
code:
GET /stuff
[{"id": 1, "name": "blah", "cost": <something that requires a lot of joins>},..]
code:
GET /stuff/1
{"id": 1, "name": "blah", "cost": <something that requires a lot of joins>}
The "cost" field, when called for the entire stuff collection, would take too long to return in a reasonable amount of time, but for a single entry isn't too bad. I'm trying to figure out if it's a better practice to keep the JSON schema for the two queries the same and have cost be null for the collection query or to remove the cost field entirely from the collection JSON or if neither is a good idea and I need to figure something else out.

Edit: That "something else" may be to redesign the tables so cost can be included in the collection query and have it perform reasonably well, but that will be quite difficult. I understand, though, that's probably the best overall solution.


If people don't use the API call to look up the "cost" then you shouldn't include it as part of the JSON that it returns. In fact if it is that computationally long to calculate, it would be better off if you created a cost call like, GET /cost/:id, that takes in an ID and returns the cost of that ID.

If the cost field is important to the call for /stuff/ then you need to figure out a way to cache the result of cost so its not constantly having to hit the database and join tables.

It's hard to suggest what the best route to take because you are the one looking at the code. If its not really needed and is only required on occasion then I would put it out on its own separate API call. Otherwise you just need to figure out how to cache the value of cost.

Gilg
Oct 10, 2002

Thank you for the responses nielsm, ComptimusPrime, and Strong Sauce.

To expand further, unfortunately it's not just the cost field that is proving difficult, it was just an example. To provide more context, this API is supporting a summary/detail view, where users first see a list of items (retrieved by the GET /stuff call) and then can choose specific ones to see in detail. The original goal was to have all the fields necessary for the summary and detailed views come from one query, so users could switch between items without any delay from querying. Due to a few detailed view fields being too expensive to query for the entire collection, however, we are going to break it up into summary view data from GET /stuff and detailed view data from GET /stuff/:id.

That's how I ended up with my null vs removing JSON fields question. Reading it again, it makes sense the two calls should return different JSON schemas.

TheMover
Jun 24, 2010

Otto Skorzeny posted:

PIC asm is kind of gnarly but workable; this being said, there are two C compilers for PIC in common use that would make your life much easier (one by Microchip and the other by CCS, if I recall correctly).

Thankyou very much, I'll certainly look into that then, anything to make life easy. Hopefully I'll make something other than blinking LEDs!

ToxicFrog
Apr 26, 2008


The Born Approx. posted:

Is there a thread for parallel programming questions? Specifically I am having issues running OpenMPI executables on a quad core machine, where it doesnt seem to acknowledge that I have multiple slots on the CPU. Any OpenMPI users around here?

As far as I know there's no thread specifically for parallel programming.

I do use openMPI, but only by way of Pilot. I've never had problems with it detecting multiple cores, but I don't know if this is down to my openMPI configuration or if there's something special that needs to be done on startup that Pilot handles for me automatically.

Adbot
ADBOT LOVES YOU

the talent deficit
Dec 20, 2003

self-deprecation is a very british trait, and problems can arise when the british attempt to do so with a foreign culture





Gilg posted:

I've got a web API that returns JSON like so
code:
GET /stuff
[{"id": 1, "name": "blah", "cost": <something that requires a lot of joins>},..]
code:
GET /stuff/1
{"id": 1, "name": "blah", "cost": <something that requires a lot of joins>}

return instead:

code:
GET /stuff
[{"id": 1, "name": "blah", "cost": {"uri": "/stuff/1"}}, ...]

GET /stuff/1
{"id": 1, "name": "blah", "cost": <something...>}

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply