Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
alo
May 1, 2005


Bob Morales posted:

...
but I wasn't sure how to get just the filename without the extension, to append .tiff to

http://www.tldp.org/LDP/abs/html/parameter-substitution.html

Look for Substring replacement, you'll want to do something like ${1/pdf/tiff}

Adbot
ADBOT LOVES YOU

Hollow Talk
Feb 2, 2014

Bob Morales posted:

I have a couple nested directories full of PDF files that need to be TIFF files to be sent somewhere.

SomeClient -
2011
commercial
residential
2012
commercial
residential
...

I know I can just use the 'convert' command on a single file:
code:
convert document001.pdf document001.tiff
What's the best way to loop through these, and rename them all?

A quick bash script with a for loop?

I started doing something like
code:
find * -name "*.pdf" -exec convert $1 xxxxxxxx {} \;
but I wasn't sure how to get just the filename without the extension, to append .tiff to

If you have zsh, you can also use that, because I find its substitution syntax really intuitive (it uses sed syntax). That way you could do something like this:
code:
for file (**/*.pdf) convert $file $file:s/pdf/tiff/
That's just making use of zsh's abbreviated for function, and is equivalent to for file in **/*.pdf ; do convert $file $file:s/pdf/tiff/ ; done.

edit: The latter should work in bash as well, when changed to bash substitution syntax you'll get: for file in **/*.pdf ; do convert $file ${file/pdf/tiff} ; done

Hollow Talk fucked around with this message at 15:17 on Mar 20, 2014

Bob Morales
Aug 18, 2006


Just wear the fucking mask, Bob

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

Ended up doing:

code:
for f in $(find . -wholename "*.pdf"); do
  convert $f ${f/%.pdf/.tiff}
  echo '.'
done

Illusive Fuck Man
Jul 5, 2004
RIP John McCain feel better xoxo 💋 🙏
Taco Defender
Any ideas on what would cause a tap device created with 'ip tuntap add dev tap1 mode tap' and attached to a bridge to be non-functional? It is not receiving or sending any packets/bytes according to ifconfig, even though the bridge is getting stuff from the attached physical device.

essentially I have:
code:
~ # brctl show                                                                  
bridge name     bridge id               STP enabled     interfaces                                  
br1             8000.001122334455       no              eth1                    
                                                        tap1
~ # ifconfig br1                                                                
br1       Link encap:Ethernet  HWaddr 00:11:22:33:44:55                         
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1            
          RX packets:776 errors:0 dropped:0 overruns:0 frame:0                  
          TX packets:25 errors:0 dropped:0 overruns:0 carrier:0                 
          collisions:0 txqueuelen:0                                             
          RX bytes:31521 (30.7 KiB)  TX bytes:2406 (2.3 KiB)
~ # ifconfig tap1                                                               
tap1      Link encap:Ethernet  HWaddr FA:16:3E:12:DE:0D                         
          UP BROADCAST MULTICAST  MTU:1500  Metric:1                            
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0                    
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0                  
          collisions:0 txqueuelen:500                                           
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
edit: I've noticed that the tap device shows NO-CARRIER when I do 'ip link ls' -- still haven't found the reason
edit2: ended up recompiling kernel with macvlan support and using that instead. I guess that was more what I needed anyway.

Illusive Fuck Man fucked around with this message at 18:04 on Mar 20, 2014

Dameius
Apr 3, 2006
Alright Linux guys, I'm trying to teach myself awk by condensing things into a single awk that was previously using multiple greps.

The script I've been revising pulls MySQL credentials out of the wp-config.php file and assigns name, user, pass, et al to shell variables for use in various WordPress management stuff.

My first draft was using

code:
db=`grep -i db_name wp-config.php |awk -F"'" '{print $4}'`
then
code:
db=$(grep -i db_name wp-config.php |awk -F"'" '{print $4}')
then
code:
db=$(awk -F"'" '/DB_NAME/{print $4}' wp-config.php)
At this point I figure I could just have awk do a single pass using regex to pull DB_NAME DB_USER and DB_PASSWORD from the config file and pass the awk output to multiple shell variables.

Great, awesome, after going back and relearning the RegEx I already forgot I end up with:

code:
cpuser@server $ awk -F "'" '/DB_[N|U|P]/{print $4}' wp-config.php
cpuser_dbname
cpuser_dbuser
password
However now I can't figure out how to direct the output from this pass to multiple shell variables, e.g. $db, $user $pass so that I could then pipe to:

code:
mysql -u $user -p$pass $db -e "MySQL stuff;"
I've been Googling for passing multiple lines of output from a single awk to multiple shell variables but I'm either not finding it or I'm not recognizing the answer in front of me. Anyone have any thoughts?

evol262
Nov 30, 2010
#!/usr/bin/perl

Dameius posted:

Alright Linux guys, I'm trying to teach myself awk by condensing things into a single awk that was previously using multiple greps.

The script I've been revising pulls MySQL credentials out of the wp-config.php file and assigns name, user, pass, et al to shell variables for use in various WordPress management stuff.

My first draft was using

code:
db=`grep -i db_name wp-config.php |awk -F"'" '{print $4}'`
then
code:
db=$(grep -i db_name wp-config.php |awk -F"'" '{print $4}')
then
code:
db=$(awk -F"'" '/DB_NAME/{print $4}' wp-config.php)
At this point I figure I could just have awk do a single pass using regex to pull DB_NAME DB_USER and DB_PASSWORD from the config file and pass the awk output to multiple shell variables.

Great, awesome, after going back and relearning the RegEx I already forgot I end up with:

code:
cpuser@server $ awk -F "'" '/DB_[N|U|P]/{print $4}' wp-config.php
cpuser_dbname
cpuser_dbuser
password
However now I can't figure out how to direct the output from this pass to multiple shell variables, e.g. $db, $user $pass so that I could then pipe to:

code:
mysql -u $user -p$pass $db -e "MySQL stuff;"
I've been Googling for passing multiple lines of output from a single awk to multiple shell variables but I'm either not finding it or I'm not recognizing the answer in front of me. Anyone have any thoughts?

Bash doesn't do this. Read the variables into an array. Or use awk's system() and skip piping them.

Or use a Perl oneliner. Awk's great, and it's amazingly flexible. Full awk scripts are magical. But it's practically a dead language in 2014 for anything moderately complex, Perl is everywhere, has similar semantics, and is a better all-around systems language these days.

Dameius
Apr 3, 2006
Knowledge and use of awk is part of the job requirements for the promotion I am chasing so I'm forcing myself to do this in awk to learn it. I'll check out using system() or an array, thanks.

pliable
Sep 26, 2003

this is what u get for "180 x 180 avatars"

this is what u fucking get u bithc
Fun Shoe
What is the preferred way to start and stop services in Ubuntu 13.10 and up? I keep reading different things, such as /etc/init.d/<service>, and then "service <service> start/stop" as being the new hotness. Can someone clarify? Thanks!

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
/etc/init.d/foo is and has always been and will always be an implementation detail of the init system. Same with /etc/rc.d/foo. Those files may not exist for Upstart or systemd services.

The correct way to do it is service foo start.

telcoM
Mar 21, 2009
Fallen Rib

Dameius posted:

However now I can't figure out how to direct the output from this pass to multiple shell variables

Getting multiple shell variables from the output of one process without invoking extra commands is tricky.
You might already have thought about using the "read" shell command... but if you run it within a pipeline, it assigns the variables in a subshell, which goes away when the execution of the pipeline is complete.

Strictly as a learning experience, I might suggest using awk to format the MySQL credentials as a series of valid shell variable assignments, and then eval'ing the output of the awk in your shell script.

Of course, your awk would then need to be prepared to properly escape all kinds of characters, or you may end up a victim of a shell version of Little Bobby Tables.

pliable
Sep 26, 2003

this is what u get for "180 x 180 avatars"

this is what u fucking get u bithc
Fun Shoe

Suspicious Dish posted:

/etc/init.d/foo is and has always been and will always be an implementation detail of the init system. Same with /etc/rc.d/foo. Those files may not exist for Upstart or systemd services.

The correct way to do it is service foo start.

gently caress that was fast, thank you sir!

pipebomb
May 12, 2001

Dear God, what is it like in your funny little brains?
It must be so boring.
Hey guys.
I need to recursively append to the wp-config.php file in multiple directories on a server. Basically, I need to do this:

* Find wp-config.php
* Make wp-config.php.back
* Insert this text at the end of the file:
code:
/** Allow updates without FTP credentials */
define('FS_METHOD','direct'); 
* Output results to a file as to which files were touched (with path)


Any thoughts on doing this safely?

evol262
Nov 30, 2010
#!/usr/bin/perl

pipebomb posted:

Hey guys.
I need to recursively append to the wp-config.php file in multiple directories on a server. Basically, I need to do this:

* Find wp-config.php
* Make wp-config.php.back
* Insert this text at the end of the file:
code:
/** Allow updates without FTP credentials */
define('FS_METHOD','direct'); 
* Output results to a file as to which files were touched (with path)


Any thoughts on doing this safely?

code:
find /path/to/user/roots -name "wp-config.php" | while read filename; do cp $filename ${filename}.bak; \
echo -e "/** Allow updates without FTP credentials */\ndefine('FS_METHOD', 'direct');" >> $filename; echo $filename >> output_file; done

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

pliable posted:

gently caress that was fast, thank you sir!

In my experience, every Upstart or systemd service puts a script in init.d that exists to tell you to use Upstart or systemd. I assume that won't always be the case, but at least for now you can use init.d scripts and if you should use something else it will tell you.

evol262
Nov 30, 2010
#!/usr/bin/perl

FISHMANPET posted:

In my experience, every Upstart or systemd service puts a script in init.d that exists to tell you to use Upstart or systemd. I assume that won't always be the case, but at least for now you can use init.d scripts and if you should use something else it will tell you.

You're gonna have so much fun when RHEL7 comes out.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

evol262 posted:

You're gonna have so much fun when RHEL7 comes out.

I'm gonna have so much fun never touching RHEL professionally in the near or medium term future. We're 100% Ubuntu.

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe

FISHMANPET posted:

In my experience, every Upstart or systemd service puts a script in init.d that exists to tell you to use Upstart or systemd. I assume that won't always be the case, but at least for now you can use init.d scripts and if you should use something else it will tell you.

Note that executing the init.d scripts directly is wrong. init.d is an implementation detail of sysvinit. The service will inherit things like envvars, resource limits, umasks, audit from your personal shell. service cleans all of this up for you.

pipebomb
May 12, 2001

Dear God, what is it like in your funny little brains?
It must be so boring.

evol262 posted:

code:
find /path/to/user/roots -name "wp-config.php" | while read filename; do cp $filename ${filename}.bak; \
echo -e "/** Allow updates without FTP credentials */\ndefine('FS_METHOD', 'direct');" >> $filename; echo $filename >> output_file; done

You are awesome. Can I send you a cuppa?

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

Suspicious Dish posted:

Note that executing the init.d scripts directly is wrong. init.d is an implementation detail of sysvinit. The service will inherit things like envvars, resource limits, umasks, audit from your personal shell. service cleans all of this up for you.

Prior to things like systemd and Upstart existing I'd never see any other way to modify services, so I'm not really sure I understand what you're saying here. Currently in Ubuntu if you use an init.d script to modify a service that's under the control of service it will just execute the proper service command and tell you to use service in the future. For example, sudo /etc/init.d/apache2 restart would execute service apache2 restart and then tell you to use service apache2 restart in the future. Without something like service I know of no other way to start and restart services, so if that's wrong then what's right?

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
I'm saying that service httpd start is right and /etc/init.d/httpd start is wrong. Always has been that way, even during sysvinit days.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
I've literally never heard that, and every guide on doing anything in Linux I've ever read until recently has instructed to use /etc/init.d/blah.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

FISHMANPET posted:

I've literally never heard that, and every guide on doing anything in Linux I've ever read until recently has instructed to use /etc/init.d/blah.
You've been following Red Hat guides without this best-practice information for awhile then, because the process you're describing literally hasn't worked on Ubuntu for about five years.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
I've never read a Red Hat guide on anything because I've never touched anything Red Hat derived. I just restarted Apache on an Ubuntu 12.04 Server with /etc/init.d/apache2 restart and it restarted apache, and the screen output was the same as doing service apache2 restart (with the exception of spitting out a PID at the end).

I also just looked and I could have sworn I wasn't able able to use service to restart postgresql, but I guess I can. But I'll reiterate, I've literally never seen this until right now, every guide on doing anything ever says to use /etc/init.d/blah.

E: I'm clearly wrong about this, and you guys are right, I've just never heard of this until now.

FISHMANPET fucked around with this message at 18:51 on Mar 21, 2014

hifi
Jul 25, 2012

FISHMANPET posted:

I've never read a Red Hat guide on anything because I've never touched anything Red Hat derived. I just restarted Apache on an Ubuntu 12.04 Server with /etc/init.d/apache2 restart and it restarted apache, and the screen output was the same as doing service apache2 restart (with the exception of spitting out a PID at the end).

I also just looked and I could have sworn I wasn't able able to use service to restart postgresql, but I guess I can. But I'll reiterate, I've literally never seen this until right now, every guide on doing anything ever says to use /etc/init.d/blah.

Well now you know better.

To contribute, there's a README file in /etc/init.d (and /var/log for that matter) that tells you what you should be using on f20, so I assume it's going to be there on rhel7.

Riso
Oct 11, 2008

by merry exmarx
You don't even start services in FreeBSD with anything but service <name> start. Using /etc/whatever is seriously deprecated.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
Also, is this a Linux thing, or a Red Hat thing? Because I've never touched Red Hat, and I won't be touching Red Hat for the foreseeable future either.

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
/sbin/service is standardized by LSB, and has always been the correct way to start services.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
This official Ubuntu documentation implies that in a land before Upstart, there was no service command, only /etc/init.d:
https://help.ubuntu.com/community/UbuntuBootupHowto

E: Wiki, so I guess not super official. But I guess I don't understand why if /etc/init.d has never been the correct way to start services, why its use is so pervasive?

evol262
Nov 30, 2010
#!/usr/bin/perl

FISHMANPET posted:

Also, is this a Linux thing, or a Red Hat thing? Because I've never touched Red Hat, and I won't be touching Red Hat for the foreseeable future either.

It's an everything thing. FreeBSD doesn't recommend /etc/rc.d or /usr/local/etc/rc.d. Debian and Ubuntu don't recommend /etc/init.d. Red Hat doesn't recommend /etc/init.d. Or Arch. Or Gentoo. Or...

You're not forced to use systemctl even on Fedora. And may never be. But /etc/init.d/${service} is not somehow a script which calls /sbin/start ${service} (on Upstart) or systemctl start ${service} (on systemd) or...

It's a path to very old scripts. It's currently present on Ubuntu because Debian was in the middle of the great init system debate, and required sysvinit, upstart, and systemd scripts to be written by maintainers. And they may always. But it doesn't operate the same way, and "service" is a shell script which tries to be smart about it. You're better off running it through upstart or systemd than sysvinit anyway for a number of reasons, and "service" makes sure you can address all of your services from the same place with a preferred search order.

hackedaccount
Sep 28, 2009
Part of the problem too is you'll talk to or read stuff on the Internet written by old graybeards like me and that knowledge keeps getting passed around. /etc/init.d/foo stop WAS correct back in the Solaris 7 days, it worked when I started learning Linux, so I just stuck with it until I stumbled across service foo stop.

You might (or might not) be surprised at how many Big Corps roll out new projects in 2014 that still use Expect and FTP when they could be using scp/rsync/whatever because knowledge the of the modern, proper ways to do stuff doesn't exist at their company. Or it's in another department that you don't know exists because...Big Corps.

hackedaccount fucked around with this message at 19:25 on Mar 21, 2014

evol262
Nov 30, 2010
#!/usr/bin/perl

FISHMANPET posted:

This official Ubuntu documentation implies that in a land before Upstart, there was no service command, only /etc/init.d:
https://help.ubuntu.com/community/UbuntuBootupHowto

E: Wiki, so I guess not super official. But I guess I don't understand why if /etc/init.d has never been the correct way to start services, why its use is so pervasive?

https://bugzilla.redhat.com/show_bug.cgi?id=44857

The service command has literally existed longer than Ubuntu, and it is not just a Redhat thing, just that it's a convenient bug to show you.

Use of /etc/init.d is pervasive because sysvinit scripts lived there and the only real other competing system (on Linux) was OpenRC, which also used /etc/init.d to avoid confusing people. So people just got used to services living there, and forgot there were other ways to do it. Some UNIXes had them in /sbin/rc.d, and some in /sbin/init.d, and *BSD still has them split between /etc/rc.d and /usr/local/etc/rc.d, and others didn't symlink them back so you had to look at /etc/rc${level}.d/ if you knew which runlevel it came from, etc.

init.d is not and has never been a standard.

alo
May 1, 2005


evol262 posted:

init.d is not and has never been a standard.

Except in the LSB. http://refspecs.linuxbase.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/etc.html

iirc, Debian only started having a service command in Sarge (~2005). I can't remember ever using invoke-rc.d. Usually /etc/init.d/service is easier due to tab completion (yes, I realize a properly set up shell can have completion with service name start). That said, since the recent shake-up of init systems, I've been using service... and probably systemctl in the future.

ps, can you link me to where /sbin/service is standardized in the LSB?

evol262
Nov 30, 2010
#!/usr/bin/perl

alo posted:

Except in the LSB. http://refspecs.linuxbase.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/etc.html

iirc, Debian only started having a service command in Sarge (~2005). I can't remember ever using invoke-rc.d. Usually /etc/init.d/service is easier due to tab completion (yes, I realize a properly set up shell can have completion with service name start). That said, since the recent shake-up of init systems, I've been using service... and probably systemctl in the future.

ps, can you link me to where /sbin/service is standardized in the LSB?

This is misleading.

The LSB requires that the directory be present, and that facilities to run scripts present in init.d be on the system, as well to install those scripts with install_initd, and that those services respond to a given set of commands. It does not require that all service files be in init.d. It does not require /sbin/service (/sbin/service makes sure that services respond to what the LSB requires and nothing more).

Salt Fish
Sep 11, 2003

Cybernetic Crumb

pipebomb posted:

You are awesome. Can I send you a cuppa?

I left you a message in the webhosting thread about the probable cause of this. Let me know if you want some assistance.

JHVH-1
Jun 28, 2002
I'm late to the party, but sed has an -i option that you can give it an extension to automatically back up files, so if you want to easily add a line or replace text you can do that and back up with one command.

sed -i.backup '$a/Some stuff to add to the end of the file'

The $ sign is for the end of the file, and the a is for append. I find it a lot cleaner than copying files and doing all that other voodoo when you can just give one command you throw in a loop or pass to find.

YouTuber
Jul 31, 2004

by FactsAreUseless
Can anyone explain why I have to compile ffmpeg from source if I want, you know, ffmpeg? Avconv loving sucks compared to ffmpeg but installs when you apt-get ffmpeg. Is there a reason why this inferior program is camping on the ffmpeg name in the Debian/Ubuntu repositories?

Hollow Talk
Feb 2, 2014

YouTuber posted:

Can anyone explain why I have to compile ffmpeg from source if I want, you know, ffmpeg? Avconv loving sucks compared to ffmpeg but installs when you apt-get ffmpeg. Is there a reason why this inferior program is camping on the ffmpeg name in the Debian/Ubuntu repositories?

I presume this has to do with licenses and the lawsuit potential of some codecs that are patent-encumbered. Debian is notoriously special when it comes to licenses. I know that openSUSE does something similar, where you are not allowed to build the full ffmpeg (as well as some other software) on the official SuSE buildservice servers (OBS), which is why there is a packman repository that usually goes alongside each openSUSE version and that holds many of the media stuff, as well as proprietary things like flash-player.

For a list of blacklisted applications for openSUSE for legal reasons: http://en.opensuse.org/openSUSE:Build_Service_application_blacklist

Debian probably has a similar list, though I do not know where you might be able to find that.

YouTuber
Jul 31, 2004

by FactsAreUseless

Hollow Talk posted:

I presume this has to do with licenses and the lawsuit potential of some codecs that are patent-encumbered. Debian is notoriously special when it comes to licenses. I know that openSUSE does something similar, where you are not allowed to build the full ffmpeg (as well as some other software) on the official SuSE buildservice servers (OBS), which is why there is a packman repository that usually goes alongside each openSUSE version and that holds many of the media stuff, as well as proprietary things like flash-player.

For a list of blacklisted applications for openSUSE for legal reasons: http://en.opensuse.org/openSUSE:Build_Service_application_blacklist

Debian probably has a similar list, though I do not know where you might be able to find that.

Avconv is literally forked directly from ffmpeg though and uses mostly all of the same patents. Just that ffmpeg patches more frequently, has compatibility for avconv and all around runs better than avconv. Avconv on the other hand doesn't have compatability for new ffmpeg functions and runs like a pig in poo poo. I highly doubt that avconv literally rewrote the entire codebase from scratch in 2 years.

JHVH-1
Jun 28, 2002
http://blog.pkh.me/p/13-the-ffmpeg-libav-situation.html

Adbot
ADBOT LOVES YOU

Longinus00
Dec 29, 2005
Ur-Quan

YouTuber posted:

Can anyone explain why I have to compile ffmpeg from source if I want, you know, ffmpeg? Avconv loving sucks compared to ffmpeg but installs when you apt-get ffmpeg. Is there a reason why this inferior program is camping on the ffmpeg name in the Debian/Ubuntu repositories?

It's because the maintainer for the debian packages is on libav's side of the whole ffmpeg/libav debacle. To be fair, before the ffmpeg/libav bifurcation you more or less had to compile ffmpeg from source to get features like multithreaded h264 handling.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply