Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Dessert Rose
May 17, 2004

awoken in control of a lucid deep dream...
a library is something you call,

a framework calls you

Adbot
ADBOT LOVES YOU

Volte
Oct 4, 2004

woosh woosh
I started making a webapp in Python without a framework by making it a series of services that don't even know about HTTP. It'll render templates, do DB access, sessions, logging in, user accounts, etc, but without HTTP. Plus I can unit test it fully without having to use special HTTP-aware tests. Then I can write a minimalist Flask wrapper for it and I don't even have to use any middleware. Specific bugs can be easily isolated to either the HTTP layer or the application layer. No more loving around with HTTP issues until it's completely necessary.

It's not not-invented-here syndrome as much as it is my frustration with the way web apps tend to be designed tightly around an HTTP framework, even down to the project structure. It does not make for an elegantly designed app and it has lead to burn-out in the past when I get sick of dealing with whichever framework I am using and don't have to time to extract and reimplement whichever part of it is frustrating me. As far as I'm concerned, HTTP is not a platform on which to build an application, but a transport protocol, with which to transport hypertext. But this is a controversial and unpopular opinion. Feels like making a VB6 program and putting all your logic right in the buttons.

karms
Jan 22, 2006

by Nyc_Tattoo
Yam Slacker

Volte posted:

I started making a webapp in Python without a framework by making it a series of services that don't even know about HTTP. It'll render templates, do DB access, sessions, logging in, user accounts, etc, but without HTTP.
All the good web frameworks I've worked with did this as well (or at least has the facilities to), but it mostly comes down to the individual programmer's discipline to separate out the transporting of the thing with the creating of the thing.

Symbolic Butt
Mar 22, 2009

(_!_)
Buglord

Volte posted:

I started making a webapp in Python without a framework by making it a series of services that don't even know about HTTP. It'll render templates, do DB access, sessions, logging in, user accounts, etc, but without HTTP. Plus I can unit test it fully without having to use special HTTP-aware tests. Then I can write a minimalist Flask wrapper for it and I don't even have to use any middleware. Specific bugs can be easily isolated to either the HTTP layer or the application layer. No more loving around with HTTP issues until it's completely necessary.

It's not not-invented-here syndrome as much as it is my frustration with the way web apps tend to be designed tightly around an HTTP framework, even down to the project structure. It does not make for an elegantly designed app and it has lead to burn-out in the past when I get sick of dealing with whichever framework I am using and don't have to time to extract and reimplement whichever part of it is frustrating me. As far as I'm concerned, HTTP is not a platform on which to build an application, but a transport protocol, with which to transport hypertext. But this is a controversial and unpopular opinion. Feels like making a VB6 program and putting all your logic right in the buttons.

idk isn't the model view controller separation usually enough for this?

like in ~~an ideal webapp~~ the model should never touch http, it's a controller thing

Sinestro
Oct 31, 2010

The perfect day needs the perfect set of wheels.
i mean, i've written code that's definitely 'model layer' that acts as a http client, but yeah

Volte
Oct 4, 2004

woosh woosh
Even the MVC frameworks are still wrapped up in the GET/POST idioms of HTTP in parts. You can definitely write a service layer, but the HTTP layer often handles too much business stuff, like handling what user is logged in. That's got nothing to do with HTTP. I don't consider HTTP to be a fundamental part of a web app, it's only the entry point to the application. I'd put code into the HTTP layer for the same reason I'd put it into the main() function. I'm a fan of designing applications as libraries with thin entry points, and most web frameworks seem like one monolithic entry point. Even if you write everything in a service layer, the structure of the application is generally dominated by the underlying framework, and that framework is almost always specifically an HTTP framework. I dunno if it's a practical difference or just an icky feeling but every single time I try to design a web app like that it ends up feeling poorly designed after awhile, and I'm pretty sure it's because I can't achieve the separation of my entire app (even the underlying session/user/db stuff) into a library with a thin HTTP entry point. My code either has to build a redundant second layer on top of all that stuff (cookies, sessions, auth, etc) to abstract it out, or let my library talk about it directly, thus coupling my environment-agnostic business logic to an arbitrarily specific entrypoint. So why not just take that redundant second layer and make it the only layer (as a library), and then use the smallest subset of Flask possible to make it into an HTTP app? It achieves my goal at least, and mirrors the relatively common pattern of implementing client applications as libraries with thin entry points.

My thing is basically just an experiment, and could end up unmanageable, but I think if I handle it right it will be more pleasant to deal with than a traditional web app tends to be. To my mind at least. Then again, I have the NIH "benefit" of only needing to implement the exact subset of features that I will ever need.

edit: Basically the 'C' in MVC has nothing to do with HTTP, which is where it usually lives. I just want to make the C completely neutral and move the HTTP in front of the whole thing. You could make a Telnet frontend too if you want.

Volte fucked around with this message at 21:32 on Sep 21, 2015

Shaggar
Apr 26, 2006
use asp.net mvc

Bloody
Mar 3, 2013

Shaggar posted:

use asp.net mvc

ahmeni
May 1, 2005

It's one continuous form where hardware and software function in perfect unison, creating a new generation of iPhone that's better by any measure.
Grimey Drawer

AWWNAW posted:

i like writing tests for poo poo when i know what the results should be but i have no idea how to implement

today i gave up trying to solve this dumb problem after getting 90% there:
you have a list of objects, each with a int property that specifies how they should be ordered
the objects are already ordered
one of them gets moved to a diff position
you gotta adjust the order properties of the list objects so that the list remains ordered, while making the least amount of changes
if you have gaps between the order ints, most of the time you only need to change the one that moved, e.g. splitting the difference between the one before and after
but if you move poo poo hundreds of times, the gaps eventually close and then you have to change a bunch of neighboring order properties
so i was trying to come up with a solution that minimized the # of changes
i think the right answer is to find the moved object and use some sort of approach that buffers the gaps between neighboring objects in both directions, hopefully not having to change more than 2-3

anyway i had a good test so i knew my solution wasn't complete and it made me feel stupid

I am not a good programmer but can this be done with a binary search tree

AWWNAW
Dec 30, 2008

ahmeni posted:

I am not a good programmer but can this be done with a binary search tree

yeah you can definitely represent the list/array as a binary tree, but that doesn't automatically solve the problem of balancing the "gap" between each sequence int and minimizing the number of sequence ints that have to be changed

or does it, and i'm just an even worse programmer?

AWWNAW
Dec 30, 2008

i'm not sure if working with it as a tree would even make it easier? seems like it might make it harder to deal with in tree form than a linear representation because of the multi-level traversals you'd have to do to ensure proper gapping between the node values

raminasi
Jan 25, 2005

a last drink with no ice

crazypenguin posted:

frameworks, probably.

like, i dunno how to define a "framework" as a distinct thing from an ordinary library except as a "pre-existing structure of interfaces and different modules implementing these interfaces that are composed via some form of DI so you can hook into it and customize whatever."

ok so i googled framework, and yeah, I like my definition better than wikipedia's.

was it tef who defined a framework as an application with the functionality removed the but the assumptions left in, or was he quoting someone, or am i totally misremembering

e: found it

raminasi fucked around with this message at 23:48 on Sep 21, 2015

MononcQc
May 29, 2007

The underlying question is after whether you're after an accurate representation of the order when the int is modified at all times, or if you want the int to be representative of the ordering -- when an int is resized, you want all other ints to be modified to represent the current order.

Volte
Oct 4, 2004

woosh woosh

Shaggar posted:

use asp.net mvc
controllers are still HTTP entry points, so what's different about that

Shaggar
Apr 26, 2006
they don't have to be if you don't want and you can write your controllers w/out any http related stuff if you really want. then put any http related stuff in the http parts of the pipeline.

Volte
Oct 4, 2004

woosh woosh
sounds cool. I would use that if using .NET was in any way shape or form an option.

AWWNAW
Dec 30, 2008

MononcQc posted:

The underlying question is after whether you're after an accurate representation of the order when the int is modified at all times, or if you want the int to be representative of the ordering -- when an int is resized, you want all other ints to be modified to represent the current order.

i want the int to be representative of the ordering. these are "objects" that have an int property that specify how they should be ordered. whenever an object gets "moved" to a different position, i want to update that sequence int to reflect its proper place in the whole thingie. the catch is how to most efficiently maintain the gaps between each object's sequence values to ensure the minimum number of changes that have to be made to other objects in the sequence. if the sequence ints are consecutive (1,2,3,4,5) and you move the last item to the front of the list, then i imagine the following options
1. decrement the moved object's sequence (== 0), only requiring the moved object to be "touched"
2. swap the moved object's sequence with the one that replaced it positionally, requires two touches
3. gap the sequences with some arbitrary size so that if you have a list like (100,200,300) and you move the last item to the middle, you end up with (100,150,200) and you only have to touch one object

#1 sucks cuz eventually you underflow or overflow
#2 sucks for inserting new items without having to consult the whole list of thingies
#3 seems like the best approach to me but you'd have to occasionally "rebalance" the gaps between neighboring objects cuz they'd eventually become consecutive numbers

AWWNAW
Dec 30, 2008

i know this has to be a solved problem but i can't find it on stack overflow, like maybe it'd be called Ol' Smitty's linear distribution rebalancing algorithm or something

Shaggar
Apr 26, 2006
that seems like a weird problem and wouldn't just moving it in the first place involve lots of poo poo getting 'touched'?

bobbilljim
May 29, 2013

this christmas feels like the very first christmas to me
:shittydog::shittydog::shittydog:
updating every single list index is O(n) anyway so not that bad. maybe people haven't bothered doing a faster one because how many times per second are you going to reorder your list of things because there's not even a case where you have to update every single list index because the worst case for moving one element is like O(n/2)

e: because because because because

AWWNAW
Dec 30, 2008

yeah it's a weird problem and i already went with a different solution that sidesteps all this poo poo, but i still want to solve this stupid toy problem

the whole point is minimizing the the # of objects that need to be "persisted" because their sequence number changed, it's not about bike shedding in-memory performance, it's about avoiding expensive serialization

Shaggar
Apr 26, 2006
why would moving objects result in serialization? or are these things stored as text and you are deserializing them to get the sequence number and reserializing them after changing it because if so then lol

MononcQc
May 29, 2007

this is roughly equivalent to knowing your position in the list, i.e. how many elements come before or after you.

If you store it in the object, then each update requires O(n) changes (without the skip mechanism) but the value is retrieved in O(1) once you got the element.

If you don't store the int, you can instead calculate it dynamically as you go through the list/array for retrieval. If your item retrieval is already O(N), then there's no additional cost (in the grand scheme of things) to doing it dynamically that way rather than storing it in each object individually.

If you use a tree, you can change that retrieval / update to O(log N), but every insertion requires you to update the parent node to increment/decrement the count on each change. When searching the tree for a position, each inner-node has a a count of how many elements are to the left and right, and can add and subtract them. It also gives you a quick way to get a total by fronting the cost at insertion time.

Dylan16807
May 12, 2010
about how many objects are you dealing with here, and what size can the ints be?

if you care about total changes and the moves are random, you can go a long way by just splitting the difference, maybe moving 1-2 neighbors. sometimes you won't be able to find space and you'll have to renumber everything, but you should be able to keep the average number of changes below 2.

to keep it to a small number of changes for each move, in a situation where objects keep being placed in similar locations, this is my best effort: after every move of an object to position P, look at several random objects and how far away their neighbors are on each side. the objects with the most extra space on the side opposite P get scooted over



MononcQc posted:

this is roughly equivalent to knowing your position in the list, i.e. how many elements come before or after you.
I don't really see the similarity. the object with ID 100 doesn't care if it has 95 things before it or 2, it just wants to stay at 100 and never move

you can't guarantee anything to be O(log N) because at some point you run low on available IDs and every move approaches O(N)

Dylan16807
May 12, 2010

AWWNAW posted:

2. swap the moved object's sequence with the one that replaced it positionally, requires two touches

wait now I'm confused, how is that a valid option, doesn't that ruin the object you swapped with? you move the last object to be near the front, now the old object that's still supposed to be near the front has sequence number 77348

MononcQc
May 29, 2007

Dylan16807 posted:

I don't really see the similarity. the object with ID 100 doesn't care if it has 95 things before it or 2, it just wants to stay at 100 and never move
The way I get it from the sentence "i want the int to be representative of the ordering." is that if you can just return the ordering, then the ids are fine.

Dylan16807 posted:

you can't guarantee anything to be O(log N) because at some point you run low on available IDs and every move approaches O(N)

You can if you use a dynamic scheme and all objects are stored in a tree. Then insertion is O(log N) and calculating the ID is reading the values in the tree :eng101:

Dylan16807
May 12, 2010

MononcQc posted:

The way I get it from the sentence "i want the int to be representative of the ordering." is that if you can just return the ordering, then the ids are fine.


You can if you use a dynamic scheme and all objects are stored in a tree. Then insertion is O(log N) and calculating the ID is reading the values in the tree :eng101:

I guess we're looking at different problems. I'll wait for AWWNAW to clarify things

AWWNAW
Dec 30, 2008

Dylan16807 posted:

about how many objects are you dealing with here, and what size can the ints be?

if you care about total changes and the moves are random, you can go a long way by just splitting the difference, maybe moving 1-2 neighbors. sometimes you won't be able to find space and you'll have to renumber everything, but you should be able to keep the average number of changes below 2.

to keep it to a small number of changes for each move, in a situation where objects keep being placed in similar locations, this is my best effort: after every move of an object to position P, look at several random objects and how far away their neighbors are on each side. the objects with the most extra space on the side opposite P get scooted over

I don't really see the similarity. the object with ID 100 doesn't care if it has 95 things before it or 2, it just wants to stay at 100 and never move

you can't guarantee anything to be O(log N) because at some point you run low on available IDs and every move approaches O(N)

yeah you smelling what i'm cooking

MononcQc
May 29, 2007

Demo code:

code:
-module(order).
-export([new/0, insert_at/3, read/2]).

new() -> [].

insert_at(Pos, Val, List) -> insert_at(Pos, Val, List, 0).

insert_at(Pos, Val, [H|T], Pos) -> [Val, H | T];
insert_at(Pos, Val, [H|T], Ct) -> [H|insert_at(Pos, Val, T, Ct+1)];
insert_at(_, Val, [], _) -> [Val].

read(N, List) -> read(N, List, 0).

read(N, [H|_], N) -> H;
read(N, [_|T], M) -> read(N, T, M+1).
Using these functions, inserting [{4,a},{17,z},{9,b},{125,y},{1,c},{3,x}] in order yields the list [a,c,z,x,b,y]:

code:
0    1    2    3    4    5
a -> c -> z -> x -> b -> y
Same stuff can be done with a tree for O(log N) rather than O(N) and the insertion by weight is relative to the position the item gets in the list -- the original value is lost, but then again, you're ready for that anyway.

AWWNAW
Dec 30, 2008

thank, i am undeserving of your erlang

let us end this terrible derail and discuss poo poo that isn't stupid

Max Facetime
Apr 18, 2009

comedyblissoption posted:

Essential mutable state is inherent to the problem you are trying to solve and cannot be eliminated.

Accidental mutable state can be subdivided:
  • Accidental mutable state that is necessary to achieve acceptable performance
  • Accidental mutable state that makes it easier to understand the problem
  • Accidental mutable state that makes it more difficult to understand the problem

There's a paper that discusses these issues:
http://shaffner.us/cs/papers/tarpit.pdf
Their proposed solution might suck though.

i'm halfway through the paper but this premise seems flawed to me:

quote:

In the ideal world we are not concerned with performance, and our language and infrastructure provide all the general support we desire. It is against this background that we are going to examine state and control.

Even in the ideal world we need to start somewhere, and it seems reasonable to assume that we need to start with a set of informal requirements from the prospective users.

Our next observation is that because we ultimately need something to happen — i.e. we are going to need to have our system processed mechanically (on a computer) — we are going to need formality.

So, taken together, this means that even in the ideal world we have:

            Informal requirements → Formal requirements

So, having produced the formalised requirements, what should the next step be? Given that we are considering the ideal world, it is not unreasonable to assume that the next step is simply to execute these formal requirements directly on our underlying general purpose infrastructure.

This state of affairs is absolute simplicity — it does not seem conceivable that we can do any better than this even in an ideal world.

First up, it would plainly be a lot more ideal if we could derive the formal requirements mechanically, then we could ignore the complexities of the specification phase in addition to ignoring the implementation phase!

more critical flaw is to start by ruling performance out of the equation for no good reason at all. if the performance of computing the results is of no concern then the least complicated way to achieve this is to never produce any results at all... obviously even in the ideal world the informal requirements would have to include "and the system SHALL at all times show how many units of work it has completed and how many more there's still to go until completion"

the result of these is an ideal computing platform so "powerful" and abstract that it needs to solve the halting problem before it can produce any actual results :newlol:

as Daniel J. Bernstein argues in The death of optimizing compilers, as computers evolve and become faster at performing some given task, the tasks that need to be performed evolve too. they involve processing larger amounts of data, or perhaps doing more involved computation. in the end performance has increased, the user got more done in the same amount of time and the complexity of the software went up.

thus it seems complexity is essential to software because user requirements are essential complexity and those are bounded by the performance of the hardware running the software

FamDav
Mar 29, 2008

AWWNAW posted:

the problem is minimizing the number of changes required. if all your sequence numbers converge to being consecutive then you have to change a bunch of objects. the hard part is just ensuring there are gaps between each object while minimizing the number of required changes I guess?

Are there constraints on the int property? does it have a min/max value (that isn't just the max addressable value or something like that)?

Do you need constant time random access?

Does it matter which int properties I change, besides trying to minimize it?

How big is this list anyways?

how often do you perform swaps, property lookups, etc. etc.?

you could use a balanced binary tree to partition the natural numbers recursively and keep the numbering invariant, which depending on how you implemented it could get you something like

1) swaps are always O(1) and property lookup is either O(1) or O(log N) depending on if you cache it or not.
2) findByIndex could be O(log N) if you store the left subtree size in your node. otherwise its O(N)
3) insertion is O(N) unless you don't cache the int property, in which case its O(log(N)).

so yeah, you can get everything down to O(Log N) or trade off insertion for constant in property if you only do swaps. the choice is yours!

AWWNAW
Dec 30, 2008

congrats, you all passed the interview and i'm preparing offer letters right now

AWWNAW
Dec 30, 2008

i look forward to being asked this question during a future amazon interview

FamDav
Mar 29, 2008
even better dumb idea.

if you're down with using a vector, just have the objects take a reference/pointer to the vector. then each object can compute its offset into the vector.

comedyblissoption
Mar 15, 2006

Max Facetime posted:

i'm halfway through the paper but this premise seems flawed to me:


First up, it would plainly be a lot more ideal if we could derive the formal requirements mechanically, then we could ignore the complexities of the specification phase in addition to ignoring the implementation phase!

more critical flaw is to start by ruling performance out of the equation for no good reason at all. if the performance of computing the results is of no concern then the least complicated way to achieve this is to never produce any results at all... obviously even in the ideal world the informal requirements would have to include "and the system SHALL at all times show how many units of work it has completed and how many more there's still to go until completion"

the result of these is an ideal computing platform so "powerful" and abstract that it needs to solve the halting problem before it can produce any actual results :newlol:

as Daniel J. Bernstein argues in The death of optimizing compilers, as computers evolve and become faster at performing some given task, the tasks that need to be performed evolve too. they involve processing larger amounts of data, or perhaps doing more involved computation. in the end performance has increased, the user got more done in the same amount of time and the complexity of the software went up.

thus it seems complexity is essential to software because user requirements are essential complexity and those are bounded by the performance of the hardware running the software
that's why i said the solution kind of sucked

i think it's really difficult to separate performance from implementation

their idea of this special formal requirements language resulting in executable code is a pipedream at this point imo

the ideas on managing complexity by minimizing mutable state are good springboards though for approaching problems

tef
May 30, 2004

-> some l-system crap ->

crazypenguin posted:

frameworks, probably.

like, i dunno how to define a "framework" as a distinct thing from an ordinary library except as a "pre-existing structure of interfaces and different modules implementing these interfaces that are composed via some form of DI so you can hook into it and customize whatever."

ok so i googled framework, and yeah, I like my definition better than wikipedia's.

you write code atop a library but inside a framework

pepito sanchez
Apr 3, 2004
I'm not mexican
i guess the best distinction i can think of between framework and library is in a framework you're working in it, not with it.

AWWNAW
Dec 30, 2008

hey it's all code right. lovely code to link against with which to write more lovely code. why discriminate or delineate

Adbot
ADBOT LOVES YOU

Max Facetime
Apr 18, 2009

comedyblissoption posted:

that's why i said the solution kind of sucked

i think it's really difficult to separate performance from implementation

their idea of this special formal requirements language resulting in executable code is a pipedream at this point imo

the ideas on managing complexity by minimizing mutable state are good springboards though for approaching problems

their observation about the way source code is written and organized, how a programmer has to pick one execution order even for statements that have no dependencies between them, and then a compiler having to prove that no dependencies exist to be able to choose a better execution order for them

that was spot on

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply