Locating the parent httpd error_log

Today I found myself on someone else’s Apache HTTPD server needing to locate the parent error_log (the one where apache records its startup). The server has a dizzying array of virtual hosts configured using a web of configuration files pulled in with Include directives. Using the configuration files to untangle which is the default virtual host and what it is using for ErrorLog was unfruitful.

My understanding is that the error_log I’m looking is the stderr for the parent apache process. So, punting, I used lsof to see where the parent process was writing stderr, a.k.a. file descriptor 2, which will be reported as ‘2w’ by lsof.

First lookup the parent httpd process. The server runs CentOS using the init system:

[root@xenu ~]# ps -u root | grep httpd
10226 ? 00:07:04 httpd

Using lsof, see what pid 10226 is using for stderr:

[root@xenu ~]# lsof -p 10226 |grep ' 2w'
httpd   10226 root    2w   REG  8,64     9411     49196 /files/www/virtualhost/RedProject/release1.0/logs/error_log

Another approach is to check the /proc filesystem:

[root@xenu ~]# ls -l /proc/10226/fd/2
l-wx------ 1 root root 64 Apr  1 16:47 /proc/10226/fd/2 -> /files/www/virtualhost/RedProject/release1.0/logs/error_log

Yep, that’s the one.

[root@xenu ~]# grep resuming /files/www/virtualhost/RedProject/release1.0/logs/error_log
[Tue Feb 21 12:52:55 2012] [notice] Apache/2.2.3 (CentOS) configured -- resuming normal operations

Configuring a Subversion browser for multiple repositories in Jenkins

Jobs in the Jenkins continuous integration software can be configured to link SCM commit details to a repository browser. Unfortunately the Subversion plugin (and perhaps all the SCM options) only allows one repository browser for all SVN repos in a given job (https://issues.jenkins-ci.org/browse/JENKINS-5289). My project uses two svn repos, Core and Rind, so this limitation, of course, poses an inconvenience.

Luckily, I was able to find a solution that works for my project. I noticed that the current Core revision numbers are in the 10K range and the Rind revisions are in the 50K range. Ah ha!, an opportunity for an Apache Rewrite rule.

I chose WebSVN for the repository browser and installed the WebSVN2 plugin (the WebSVN bundled with Jenkins is for the WebSVN 1.x series). For the browser URL configuration I used a fake repname (TBD) in the query string.

http://websvn.crashingdaily.com/revision.php?repname=TBD

and then used the following Apache RewriteRules to redirect to the proper repo in WebSVN based on the revision number.

RewriteEngine on
RewriteCond %{QUERY_STRING} repname=TBD&(.*)rev=([1234]\d{4,4})
  RewriteRule ^(.+)$ $1?repname=GUS&%1rev=%2 [R,L,NE]
RewriteCond %{QUERY_STRING} repname=TBD&(.*)rev=([56789]\d{4})
  RewriteRule ^(.+)$ $1?repname=ApiDB&%1rev=%2 [R,L,NE]

That is, if the rev number matches a number between 10000 and 49999 then redirect to the Core repo in WebSVN. If rev number matches a number over 50000 then redirect to the Rind repo.

In both cases the Rewrite depends on the repname being TBD so we do not trigger rewrites when normally browsing WebSVN.

This scheme depends on conditions and assumptions that happen to fit our project:

– We only care about current and future revisions (>10000). That’s ok, Jenkins will not need to link to revisions that predate this setup.

– Rind will always get more commits than Core such that Core revisions will always lag behind Rind revisions. This has been the case for the past 6 years, I don’t expect it to change.

– Core revisions will not exceed 49999 for the life of this Rewrite. If it does (unlikely, it’s taken 6 years to get to the 10K range) we can update the rules.

Obviously this is not a suitable solution for everyone.

Sidenote:

I initially started with sventon behind an Nginx proxy before deciding that WebSVN was going be easier for me to maintain in the long run (simply due to my system infrastructure and experience, not due to any complaint against sventon). For the record, this is where I was heading with the Nginx rewrite rules. I did not follow through with testing so I do not know if it works as is, but perhaps it’s a starting point for someone.

if ($request_uri ~ "revision=(.+)" ) {
  set $revision $1;
}

if ($revision ~ "[1234]\d{4}") {
  rewrite ^ http://sventon.crashingdaily.org/repos/Core/info?revision=$revision?;
}
if ($revision ~ "[56789]\d{4}") {
  rewrite ^ http://sventon.crashingdaily.org/repos/Rind/info?revision=$revision?;
} 

Using groups with TestNG

This post demonstrates a simple example of using groups with TestNG.

For this exercise I’m using a very simple file layout. The jar file was copied directly from the TestNG package. SampleTest.java and testng.xml are my test and configuration files, respectively.

[23:31 20090914 crashing@desktop ~/simpletestng]
$ ls
SampleTest.java testng-5.10-jdk15.jar testng.xml

SampleTest.java has two test methods, annotated with @Test, and with an assigned group name.

import org.testng.annotations.*;
public class SampleTest {
@Test(groups = { “groupA” })
public void methodA() {
System.out.println(“groupA”);
assert true;
}

@Test(groups = { “groupB” })
public void methodB() {
System.out.println(“groupB”);
assert true;
}
}

testng.xml defines these two groups as belonging to a test set.

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd"&gt;
<suite thread-count="5" verbose="1" name="TrialTest" annotations="JDK">

<test name="TestSet" enabled="true">
<groups>
<run>
<include name="groupA"/>
<include name="groupB"/>
</run>
</groups>

<classes>
<class name="SampleTest"/>
</classes>
</test>

</suite>

I compile the SampleTest class,

$ javac -classpath testng-5.10-jdk15.jar SampleTest.java

and then run all the test methods as configured in testng.xml

$ java -classpath .:testng-5.10-jdk15.jar org.testng.TestNG testng.xml

[Parser] Running:
/Users/crashing/simpletestng/testng.xml

groupA
groupB

===============================================
TrialTest
Total tests run: 2, Failures: 0, Skips: 0
===============================================

Both test methods ran.

Alternatively I can specify specific groups to run. For this I use the -groups and -testclass option. I do not use the testng.xml file.

java -classpath .:testng-5.10-jdk15.jar org.testng.TestNG -groups groupB -testclass SampleTest

[Parser] Running:
Command line suite

groupB

===============================================
Command line suite
Total tests run: 1, Failures: 0, Skips: 0
===============================================

Specifying both the testng.xml and a -groups option will cause both configurations to be processed.

[22:58 20090914 crashing@desktop ~/simpletestng]
$ java -classpath .:testng-5.10-jdk15.jar org.testng.TestNG testng.xml -groups groupA -testclass SampleTest
[Parser] Running:
/Users/crashing/simpletestng/testng.xml
Command line suite

groupA
groupB

===============================================
TrialTest
Total tests run: 2, Failures: 0, Skips: 0
===============================================

groupA

===============================================
Command line suite
Total tests run: 1, Failures: 0, Skips: 0
===============================================

This is probably not what you want. So, use either the testng.xml or the -groups and -testclass option, not both.

Defining test descriptions in testng reports

The output html of testng‘s reporter has the columns “Test method” and “Instance” that document each test run.

“Test method” can be customized by defining public String getTestName() method in the class providing the test method.

“Instance” can be customized by defining public String toString() method in the class providing the test method.

Running BLAST on Windows XP

Today I installed NCBI BLAST on Windows XP SP2 but was unable to run the executables – I was confronted with “the system cannot execute the specified program”.

The fix was to install Microsoft Visual C++ 2005 SP1 Redistributable Package

Persisting ssh connections while changing networks

I recently re-discovered autossh. I’ve been using it to persist mounted sshfs volumes but am only just now realizing it’s the scratch for some of my other itches.

With it I can

autossh -M50000 -t -D1080 -Nf crashingdaily.com

to keep a seemingly persistent SOCKS proxy running.

Or

autossh -M50000 -t crashingdaily.com 'screen -aAdR autossh'

to maintain an interactive session.

The value that I missed earlier is that the connections are maintained (re-established, actually) even if I change networks – which my laptop and I often do.

Jon’s View has a nice posting on using autossh for SOCKS proxying.

And Dread Pirate PJ has a good instruction for combining autossh and screen for persistent interactive sessions.

autossh is available for OS X via Darwin Ports and Fink and for Linux as source code or as binary packages from repositories for most modern Linux distributions.

Cesium for automated YSlow reporting

Hot on the heels of yesterday’s posting about my brief experience with YSlow in a Selenium framework, comes the release of Cesium 0.1, a tool specifically designed for automating YSlow runs.

I don’t care enough about YSlow tests to setup and maintain another test harness just for that purpose, so I don’t think I’ll be trying it myself. For others who get in to this sort of thing, here’s something to get in to.

Scheduling jobs with at and cgi scripts

The at command can be used in CGI scripts to remotely trigger non-recurring jobs on a web server. This is especially useful for long running jobs that clients don’t need to wait on and jobs that need to persist across web daemon restarts.

My initial simple attempts to schedule a job with at in a CGI script executed by an Apache webserver were met with the failure response “This account is currently not available“. That message being the output of /sbin/nologin which is the shell value set for the apache user in /etc/password.

The fix is to set the SHELL environment variable to a proper shell when executing at. Here’s a simple, proof-of-concept CGI script.

#!/usr/bin/perl
use CGI;
my $cgi = new CGI;
print $cgi->header;

system(‘echo “/usr/bin/env > /tmp/apacheenv” | SHELL=/bin/bash at now’);

Calling the CGI script triggers the at scheduling. In this case, the execution of /usr/bin/env is immediate and successful.

$ cat /tmp/apacheenv
SERVER_SIGNATURE=
SHELL=/bin/bash
HTTP_USER_AGENT=Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_7; en-us) AppleWebKit/528.18.1 (KHTML, like Gecko) Version/4.0 Safari/528.17
SERVER_PORT=80

Related:

at man page

Safebrowsing with Selenium

Firefox 3.0.3 (on OS X at least) launched by Selenium RC 1.0 beta 1 displays an alert dialog box:

Alert
sb-ssl.google.com:443 uses an invalid security certificate.
The certificate is not trusted because the issuer certificate has expired.
(Error code: sec_error_expired_issuer_certificate)

I’ve seen a report that the nightly build doesn’t have this issue but, as a quick fix for Se 1.0b1, one can edit /Applications/Firefox 3.0.3.app/Contents/MacOS/defaults/pref/firefox.js
and change a couple of "browser.safebrowsing" preferences to false.

pref("browser.safebrowsing.enabled", false);
pref("browser.safebrowsing.malware.enabled", false);

That disables Google Safe Browsing – a feature which isn’t important to me but maybe it is to you, in which case this solution isn’t your cup of tea.

Related

Browser.safebrowsing.enabled
Google Safe Browsing for Firefox

Apache access control and mod_rpaf

I’m using the Apache mod_rpaf module to capture client IP addresses in the X-Forwarded-For header passed by an Nginx reverse proxy. This is good for logging and CGI environments but mod_rpaf does not fix up the client IP address sufficiently to be used in Apache’s allow/deny access control directives.

Almlys has a nice workaround.

Quoting the juicy bit from Almlys’s blog posting:

SetEnvIf X-Forwarded-For ^172\.26\.0\.17 let_me_in
Order allow,deny
allow from env=let_me_in

Clever.