|
TasteMyHouse posted:you could write something that parses the script, grabbing all the strings on the right hand side of assignments to the variables you care about. It actually probably wouldn't be that hard. If you suggest something like that, make sure to mention the CPAN module PPI.
|
# ? Jul 30, 2011 08:49 |
|
|
# ? May 22, 2024 10:17 |
|
I have a list of around 5000 virtual and physical server IPs that I have to run through to determine if they automatically redirect http requests (port 80) to https requests (port 443). I've tried looking for scripts that do this, but haven't been able to find any. Most of them are how to set it up or disable it, not determine if it is happening. I'm not a coder/network guy by trade, so a lot of this stuff is new to me. Any suggestions on tools to use or how to go about writing up a perl script to do this? I know how to read in the list and kick off whatever command needs to be done, I just don't know what tools/modules could be used to figure this out.
|
# ? Aug 8, 2011 14:37 |
|
edit: nm
uG fucked around with this message at 18:03 on Aug 8, 2011 |
# ? Aug 8, 2011 16:33 |
|
Bazanga posted:I have a list of around 5000 virtual and physical server IPs that I have to run through to determine if they automatically redirect http requests (port 80) to https requests (port 443). Something like: code:
|
# ? Aug 9, 2011 11:28 |
|
Carthag posted:Something like: Why use curl? code:
|
# ? Aug 9, 2011 13:09 |
|
Roseo posted:Why use curl? I was too lazy to look up the LWP docs
|
# ? Aug 9, 2011 15:00 |
|
code:
This has worked really well so far. Thanks a ton. code:
|
# ? Aug 9, 2011 18:36 |
|
Bazanga posted:I don't know if I am doing something wrong, but the is_redirect attribute in LWP isn't getting set to true whenever a website redirects. For example, if I use the website http://www.capitalone.com/ as a url, LWP returns a success code and doesn't mention anything about a redirect, while I know for certain that the url redirects. What is actually happening is that LWP::UserAgent is following the redirects in its code path, successfully loads the redirected URL https://www.capitalone.com/ and states "Yep, that https URL loaded correctly". is_redirect is there to inform you that you are not stuck in an infinite redirect loop. To have it do what you want (no follow redirects, and inform you of whether a URL requests a redirect), instantiate LWP::UserAgent like so: code:
|
# ? Aug 9, 2011 20:12 |
|
Doesn't $response->request()->uri() return the final uri, which you could compare to $host? Or even just look at $response->redirects?
|
# ? Aug 9, 2011 21:29 |
|
homercles posted:What is actually happening is that LWP::UserAgent is following the redirects in its code path, successfully loads the redirected URL https://www.capitalone.com/ and states "Yep, that https URL loaded correctly". is_redirect is there to inform you that you are not stuck in an infinite redirect loop. Ugh, duh. Sorry about that, Bazanga.
|
# ? Aug 10, 2011 00:01 |
|
Thanks! That was incredibly useful. Once I got over the is_redirect hurdle it all started clicking. You guys are livesavers.
|
# ? Aug 10, 2011 16:51 |
|
This could work just as well in the coding horrors thread, but I'll post it here instead since I know it's something Mithaldu has dealt with. Passing arguments to other processes on Windows is unreasonably hard, and even worse in Perl. First is that argument lists are always passed as a single string in Windows, as opposed to arrays on other systems. This is less of a problem than it appears, because 95% of programs use the same rules for parsing that string into an array. Roughly speaking, the rules are that arguments can be quoted with double quotes, and backslashes can escape any character. The second issue is that cmd.exe uses different quoting rules than the normal parsing routine. It uses caret as the escape character instead of backslash. The result of this is that you can't create a string that will be treated the same for both of these cases. And since Perl tries to be 'smart' about this, you have to quote strings differently depending on if they have shell meta-characters or not. And there isn't any good way to check that without reimplementing the code to detect them that exists inside perl. So here is a routine that will quote arguments correctly to use with system: code:
I tried adapting ExtUtils::MakeMaker to use a routine like this, but there were a number of issues. First is that the tests and possibly other parts of ExtUtils::MakeMaker assume it can create a string that will work both in a makefile (which always uses cmd rules) and through system() directly (which doesn't). There's also the nmake/dmake/make quoting issues. So I gave up on fixing that. There's probably some things that Perl should do differently regarding this, but then you run into backward compatibility issues...
|
# ? Aug 23, 2011 08:54 |
|
I don't know anything about Makefiles but I ran into that exact same issue and I found it to be incredibly difficult to get to the bottom of, both for the Windows command line and in Bash. At least in Bash the solution is fairly simple: put a backslash before anything that looks funny, and don't use any quotes at all. For Windows the procedure is maddening as you just discovered. Based on the same article you pointed out, I came up with this which looks much the same. One thing that that article doesn't make entirely clear is that escaping the name of the program to run is a different matter from escaping its arguments.
|
# ? Aug 23, 2011 12:41 |
|
Is there any way to work with excel files with more than 65000 rows with Perl? Carrying over the rows into a new sheet won't work. Right now i'm saving my files as csv and then manually making each csv file a sheet in a workbook, but my file is getting so big that this laptop can't handle it. So I don't really need all the formatting options or formula stuff, just more like a tabbed csv file. edit: Excel::Writer::XLSX should work edit2: Getting out of memory errors :/ uG fucked around with this message at 07:21 on Aug 24, 2011 |
# ? Aug 23, 2011 17:05 |
|
Haarg posted:Passing arguments to other processes on Windows is unreasonably hard, and even worse in Perl. Arguments on Windows are not parsed by the shell. They are actually parsed by whatever function is compiled into the C library that receives the arguments. Meaning that HOW they are parsed can change depending on the compiler version used for the program that is being called, which broke one of the recent Windows Perl builds.
|
# ? Aug 24, 2011 11:05 |
|
Yeah, that was the first part of what I was saying, and kind of punted on the 95% doing it the same thing. I'm not surprised to hear it's worse than I was thinking. Do you have a reference about the changes between different compilers/runtimes? I'd be interested to see what was changed.
|
# ? Aug 24, 2011 17:02 |
|
Haarg posted:Do you have a reference about the changes between different compilers/runtimes? I'd be interested to see what was changed.
|
# ? Aug 24, 2011 17:58 |
|
Reading up on that, it looks like a change in how repeated double quotes are parsed, which was never a documented behavior. The code I posted uses backslashes to escape strings instead, so it should work in all cases (with a MS CRT). The existing code in ExtUtils::MakeMaker tries to create strings that will work in both cmd.exe and when running the process directly. It gets pretty close, but can never be perfect. The only real solution is to have a different behavior when using cmd.
|
# ? Aug 24, 2011 18:52 |
|
Haarg posted:Reading up on that, it looks like a change in how repeated double quotes are parsed, which was never a documented behavior. The code I posted uses backslashes to escape strings instead, so it should work in all cases (with a MS CRT). If you understand it better than i did at that time, please do fork it and provide patches. I'm way too burnt out to do it myself. I will however happily merge any pull request with some decent tests.
|
# ? Aug 24, 2011 20:25 |
|
I'd love to fix it but it involves figuring out ExtUtils::MakeMaker internals. If I can get anything working I'll bug you on IRC.
|
# ? Aug 25, 2011 08:25 |
|
uG posted:Is there any way to work with excel files with more than 65000 rows with Perl? Carrying over the rows into a new sheet won't work. Right now i'm saving my files as csv and then manually making each csv file a sheet in a workbook, but my file is getting so big that this laptop can't handle it. So I don't really need all the formatting options or formula stuff, just more like a tabbed csv file.
|
# ? Aug 25, 2011 15:50 |
|
So is there no decent way to work with large excel files with Perl? Preferably not Win32::OLE?
|
# ? Aug 25, 2011 18:44 |
|
uG posted:So is there no decent way to work with large excel files with Perl? Preferably not Win32::OLE? Have you tried Spreadsheet::WriteExcelXML? I haven't used it personally, but as long as it doesn't have the memory problems of Excel::Writer, it should do what you need.
|
# ? Aug 26, 2011 14:19 |
|
You could "cheat" and serialize to csv, then use a third party tool to convert to xlsx. There seem to be a number of those on google.
|
# ? Aug 26, 2011 14:28 |
|
Erasmus Darwin posted:Have you tried Spreadsheet::WriteExcelXML? I haven't used it personally, but as long as it doesn't have the memory problems of Excel::Writer, it should do what you need. Carthag posted:You could "cheat" and serialize to csv, then use a third party tool to convert to xlsx. There seem to be a number of those on google. I'm wondering if I can format it as html tables, if excel will import it and keep the coloring format...
|
# ? Aug 26, 2011 16:04 |
|
Haarg posted:I'd love to fix it but it involves figuring out ExtUtils::MakeMaker internals. If I can get anything working I'll bug you on IRC. If you need any info, let me know. I have no idea who you are on irc, but judging from what you said i think you know who and where i am.
|
# ? Aug 27, 2011 10:08 |
I'm trying to remove all the brace characters, '[' and ']', from a bunch of utf8 unicode text files. I just tried a simple script:code:
|
|
# ? Aug 30, 2011 15:55 |
|
My go-to is utf8::all. Install that from CPAN and add -Mutf8::all to that perl invocation.
|
# ? Aug 30, 2011 20:59 |
|
While you should probably encode/decode the text when processing it, one of the features of UTF-8 is that ASCII characters are never part of other sequences. It shouldn't be possible to mangle other characters if you are only modifying [ and ]. Are you sure the text is actually in UTF-8? And what kind of mangling are you seeing?
|
# ? Aug 31, 2011 17:03 |
|
This might be more appropriate in an apache thread, but i'll ask here first. I'm trying to get Catalyst working with apache2 and fcgid. Everything works, but it looks like its loading everything up every visit, instead of keeping connected to the same fcgi socket. This is what my vhost looks like:code:
I'm thinking I should be doing something with plack now that its packaged with Catalyst, but I don't know what. uG fucked around with this message at 04:34 on Sep 1, 2011 |
# ? Sep 1, 2011 03:21 |
|
uG posted:I'm thinking I should be doing something with plack now that its packaged with Catalyst, but I don't know what.
|
# ? Sep 1, 2011 07:48 |
|
Can someone advise me on the best way to do multi-line regex's? I'm trying to scrape a list of Perforce jobs, which output in this format...code:
My usual method for parsing information out of data like this would be to do this... code:
Any recommendations on the best way to tackle this? EDIT: This code works, but only grabs the first iteration of data. code:
syphon fucked around with this message at 19:45 on Sep 12, 2011 |
# ? Sep 12, 2011 19:14 |
|
syphon posted:Can someone advise me on the best way to do multi-line regex's? This is part of some code for cleaning up a COBOL copybook. We didn't want comments or 88-levels, which are basically lines that start with whitespace, then "88" and could continue for multiple lines until the line-end character, which is a period. So if you test on this: my $eightyeightlevel = /^\s+88/ .. /\./; it returns true when it hits the first line and matches the left side, then it continues to return true until it's matched the right side. But like I said, I really don't know how capturing parentheses would work with this. It might be worth experimenting with, though.
|
# ? Sep 12, 2011 19:59 |
|
I tested this with a cut-and-paste of the data you presented, From here: code:
code:
code:
Apsyrtes fucked around with this message at 20:15 on Sep 12, 2011 |
# ? Sep 12, 2011 20:11 |
|
Apsyrtes posted:I tested this with a cut-and-paste of the data you presented,
|
# ? Sep 12, 2011 21:10 |
|
code:
code:
What should I change to increase performance without having a bunch of running counts in globals? I don't really understand what I should be caching, and pre fetching, etc, but I have used dbix::class for awhile
|
# ? Sep 13, 2011 04:46 |
|
syphon posted:Can someone advise me on the best way to do multi-line regex's? Here's a pretty efficient way: code:
|
# ? Sep 27, 2011 16:51 |
|
uG posted:I have used dbix::class for awhile There's your problem! I don't understand this modern trend to hide an expressive language under a clunky abstraction and call it best practice.
|
# ? Sep 27, 2011 16:56 |
|
Anyone have lots of experience with IO::Socket? I'm having a weird issue where the timeout parameter isn't being honored on some systems, and I can't figure out why. If you google it, you'll find people saying that some timeout stuff is commented out in INET.PM! This seems like a likely culprit, but doesn't explain why it does work on some boxes but doesn't on others.code:
My workstation is Win7, while the lab box is Win2k3, but I'm invoking a shared install of Perl 5.8.8 (IO::Socket 1.29), so the versions are all identical between the two. I see lots of talk about using alarms to get around this, but I've never used alarms before so I'm not sure how effective that'd be. Any advice? EDIT: I should clarify that port 8008 is just a dummy port for testing that I know isn't open for either system. syphon fucked around with this message at 00:14 on Oct 7, 2011 |
# ? Oct 7, 2011 00:11 |
|
|
# ? May 22, 2024 10:17 |
|
The time it takes new to return is going to depend on a lot of things, including how the remote end responds (or doesn't respond) to closed or blocked ports and the number of retries configured. TCP is guaranteed to return eventually from connect(), but the amount of time depends on a lot of things.
|
# ? Oct 7, 2011 02:03 |