|
After converting the GNU Emacs repository to Git a few years back, Eric S Raymond has been working on the massive undertaking of transferring the GCC (GNU Compiler Collection) repository in full over to Git. But the transition to GCC Git is being hampered since due to the massive size of the repository, Raymond's system is running under extreme memory pressure with 64GB of RAM. ESR provided an update on the GCC repository conversion process. He has managed to solve the only known remaining technical bug that's been blocking the repository, but now he can't get the process completed since he's over-running memory capacity. His primary workstation has 64GB of DDR4 memory and that's turned out to not be enough for the GNU Compiler Collection repository with more than a quarter million commits over the past three decades. Due to more DDR4 memory being "hideously expensive", he's figuring out alternatives. While the GNU has the GCC compile farm with multiple systems having 128GB+ of RAM, ESR believes his system is faster and better tuned for the memory-heavy workloads. Among the measures being now attempted are running the process with his web-browser shutdown to conserve memory and potentially shifting some code from Python to the Go programming language, but that could prove to be very tedious. Eric further elaborated, "The truth is we're near the bleeding edge of what conventional tools and hardware can handle gracefully. Most jobs with working sets as big as this one's do only comparatively dumb operations that can be parallellized and thrown on a GPU or supercomputer. Most jobs with the algorithmic complexity of repository surgery have *much* smaller working sets. The combination of both extrema is hard." Some developers have talked about chipping in money so ESR can upgrade the RAM in his system and there is also the fact that the GNU/FSF does collect money for different causes (such as the FSF GNU Toolchain Fund) could potentially be tapped. We'll see what happens but he wants to act on it sooner rather than later as the GCC repository only continues getting larger by the day.
|
#
?
Jul 9, 2018 20:04
|
|
|
|
| # ? Apr 29, 2024 16:40 |
|
|
Tankakern posted:running the process with his web-browser shutdown to conserve memory well besides that i'm out of ideas. sorry esr
|
|
#
?
Jul 9, 2018 20:47
|
|
|
he should try threatening it with armed revolution if it doesn’t work the way he wants
|
|
#
?
Jul 9, 2018 20:51
|
|
|
|
|
#
?
Jul 9, 2018 20:52
|
|
|
i could never provide monetary assistance to a baby boomer, sorry.
|
|
#
?
Jul 9, 2018 21:23
|
|
|
i thought esr was busy solving hard ai problems
|
|
#
?
Jul 9, 2018 21:24
|
|
|
Tankakern posted:He has managed to solve the only known remaining technical bug that's been blocking the repository, but now he can't get the process completed 
|
|
#
?
Jul 9, 2018 21:58
|
|
|
esr wants to shoot minorities then ram his alpha cock in the awe struck sjw hotties
|
|
#
?
Jul 9, 2018 22:01
|
|
|
do it from an aws vm with tons of memory? what the gently caress is the problem here
|
|
#
?
Jul 9, 2018 23:09
|
|
|
yes let's do this all as one giant atomic operation that has to keep all file contents and history in memory
|
|
#
?
Jul 9, 2018 23:35
|
|
|
has he considered a ram doubler, op?
|
|
#
?
Jul 9, 2018 23:36
|
|
|
pram posted:do it from an aws vm with tons of memory? what the gently caress is the problem here he's a talentless retard fuckwit.
|
|
#
?
Jul 9, 2018 23:40
|
|
|
try setting pack size limit and window memory, that worked for me
|
|
#
?
Jul 9, 2018 23:56
|
|
|
he needs $2000
|
|
#
?
Jul 10, 2018 00:12
|
|
|
this is the stupidest thing ive seen all week and ive been on these forums a lot
|
|
#
?
Jul 10, 2018 00:23
|
|
|
It's now safe to turn on your monitor.
|
|
#
?
Jul 10, 2018 00:29
|
|
|
esr wants your cum
|
|
#
?
Jul 10, 2018 00:30
|
|
|
git init hth esr
|
|
#
?
Jul 10, 2018 00:32
|
|
|
why is this an O(n^2) problem and who would throw money at esr
|
|
#
?
Jul 10, 2018 00:34
|
|
|
please tell me he's going full wopr just to make another copy of this
|
|
#
?
Jul 10, 2018 00:38
|
|
|
i'm still angry at whoever posted that thread that was titled something like "eric s raymond dead in car accident at age 56" like 5 years ago which turned out to be fake news of the worst sort
|
|
#
?
Jul 10, 2018 00:45
|
|
|
DDR4 is hideously expensive? drat, I think he forgot about DDR/DDR2 pricing
|
|
#
?
Jul 10, 2018 00:53
|
|
|
also just increase your swap space
|
|
#
?
Jul 10, 2018 01:03
|
|
|
push the turbo button
|
|
#
?
Jul 10, 2018 01:42
|
|
|
CommieGIR posted:DDR4 is hideously expensive? drat, I think he forgot about DDR/DDR2 pricing rambus 
|
|
#
?
Jul 10, 2018 01:51
|
|
|
prescott p4, rdram, winme, sick pc
|
|
#
?
Jul 10, 2018 02:05
|
|
|
did somebody say ram?
|
|
#
?
Jul 10, 2018 02:14
|
|
|
Fuzzy Mammal posted:prescott p4, rdram, winme, sick pc so sick it should be shot
|
|
#
?
Jul 10, 2018 02:23
|
|
|
esr called black people alien poo poo zombies
|
|
#
?
Jul 10, 2018 02:53
|
|
|
he’s bad
OldAlias fucked around with this message at 03:04 on Jul 10, 2018 |
|
#
?
Jul 10, 2018 03:01
|
|
|
SmokaDustbowl posted:esr called black people alien poo poo zombies git rm —cache
|
|
#
?
Jul 10, 2018 03:28
|
|
|
what a loving idiot. but his idiocy aside, git is very bad for large repos. like, really really bad.
|
|
#
?
Jul 10, 2018 03:30
|
|
|
Silver Alicorn posted:rambus We don't talk about RAMBUS
|
|
#
?
Jul 10, 2018 03:41
|
|
|
Heck yeah hes a R acist A sshole M anchild
|
|
#
?
Jul 10, 2018 03:58
|
|
|
The Management posted:what a loving idiot. It would be funny if the fsf ends up having to use Microsoft gvfs to work on gcc
|
|
#
?
Jul 10, 2018 07:36
|
|
|
pram posted:do it from an aws vm with tons of memory? what the gently caress is the problem here
|
|
#
?
Jul 10, 2018 08:12
|
|
|
Im the multiple root revisions from when gnu merged all its compiler projects together
|
|
#
?
Jul 10, 2018 08:20
|
|
|
quote:Among the measures being now attempted are running the process with his web-browser shutdown to conserve memory and potentially shifting some code from Python to the Go programming language, but that could prove to be very tedious.
|
|
#
?
Jul 10, 2018 08:21
|
|
|
yospos project: let’s murder esr
|
|
#
?
Jul 10, 2018 08:31
|
|
|
|
| # ? Apr 29, 2024 16:40 |
|
|
OldAlias posted:yospos project: let’s murder esr i've got an untraceable gun and a toyota previa with half a tank of gas
|
|
#
?
Jul 10, 2018 08:37
|
|
