Safebrowsing with Selenium

Firefox 3.0.3 (on OS X at least) launched by Selenium RC 1.0 beta 1 displays an alert dialog box:

Alert
sb-ssl.google.com:443 uses an invalid security certificate.
The certificate is not trusted because the issuer certificate has expired.
(Error code: sec_error_expired_issuer_certificate)

I’ve seen a report that the nightly build doesn’t have this issue but, as a quick fix for Se 1.0b1, one can edit /Applications/Firefox 3.0.3.app/Contents/MacOS/defaults/pref/firefox.js
and change a couple of "browser.safebrowsing" preferences to false.

pref("browser.safebrowsing.enabled", false);
pref("browser.safebrowsing.malware.enabled", false);

That disables Google Safe Browsing – a feature which isn’t important to me but maybe it is to you, in which case this solution isn’t your cup of tea.

Related

Browser.safebrowsing.enabled
Google Safe Browsing for Firefox

Allowing any and all

A mod_rpaf story.

My Apache virtual hosts are dynamically configured from a library of Perl code using PerlSections.
A configuration file for one virtual looks something like:

<VirtualHost *:80>
 <Perl>
  require 'conf/lib/MasterConf.pm';
 </Perl>
</VirtualHost>

Aside from boostrapping all the basic virtual host configuration, MasterConf.pm establishes basic authentication (via mod_auth_tkt) for the <Location /> of the virtual host. Authentication is the desired default but, for some specific virtual hosts, I want to disable the authentication requirements. This I can do by appending a new <Location /> after the Perl Section.

<VirtualHost *:80>
 <Perl>
  require 'conf/lib/MasterConf.pm';
 </Perl>
 <Location />
  Allow from any
 </Location>
</VirtualHost>

This effectively overrides the deny from declared by the perl library and makes the site public. This has been working fine for me for years. Recently though, I put an Nginx reverse-proxy in front of some of my virtual hosts and installed mod_rpaf into Apache so the IP address it logged would be the client’s and not the proxy server.

There I hit a snag. Apache/mod_rpaf was ignoring the X-Forwarded-For header and only logging the IP address of the proxy. Actually the GET '/' request was logging correctly, but that was insufficient and no consolation.

Client logging was done correctly if I used a static, hand-crafted Apache configuration file (not using the Perl library) so I knew mod_rpaf was installed correctly. I began to suspect the dynamic Perl configuration and that made me nervous; I really didn’t want to give that up completely.

Well, cutting to the chase, it turns out that the ‘Allow from any’ added to disable the basic authentication was the culprit. I’ve been using ‘Allow from any’ for years and it does indeed allow any host access but looking at it anew today it suddenly struck me as being wrong. It’s normally ‘Allow from all‘, I must have had ‘Satisfy any‘ on the brain when I originally started using the override – propagating around through thoughtless copying/pasting. On a whim I changed it to ‘Allow from all‘ and, w00t!, mod_rpaf began logging the correct client address.

Are ‘allow from any’ and ‘allow from all’ supposed to mean different things to Apache? A little Googling turns up examples of people using ‘allow from any’ but I don’t see any indication that it has a special meaning.

A Workaround For Net::SFTP Hanging On Large Transfers

[update: This issue is reportedly fixed in Net::SSH::Perl 1.34]

I ran into an issue with the Perl module Net::SFTP that drives the file transfers in a script that I use. Net::SFTP is built on top of Net::SSH::Perl, a pure Perl implementation of the SSH-2 protocol.

The issue is that after 1 GB of data is transferred the connection hangs indefinitely.

I found something similar was reported before in the Net::SSH::Perl bug tracker a couple of years ago. Though in that case the hanging was after 1 hour time rather than 1 GB data transfered. The bug report was rejected by the maintainer just the other day with the comment to “try the latest version”.

Well, I have the latest version of Net::SSH::Perl and Net::SFTP from CPAN (1.30) and still have the issue.

I did some digging.

On the server side I have

OpenSSH_3.9p1, OpenSSL 0.9.7a Feb 19 2003

If I turn on sshd debugging I get this logged at the time of the transfer stalling:

Feb 15 23:21:55 olive sshd[19497]: debug1: need rekeying
Feb 15 23:21:55 olive sshd[19497]: debug1: SSH2_MSG_KEXINIT sent

The client Perl script using Net::SFTP with debugging on, reports:

sftp: In read loop, got 8192 offset 1069547520
sftp: Sent message SSH2_FXP_READ I:130563 O:1069555712
Warning: ignore packet type 20
channel 1: window 16358 sent adjust 16410
sftp: Received reply T:103 I:130563
sftp: In read loop, got 8192 offset 1069555712
sftp: Sent message SSH2_FXP_READ I:130564 O:1069563904

I did some more digging and found that the ‘packet type 20’ that is being ignored corresponds to the SSH2_MSG_KEXINIT sent by the server and is a request to initiate session rekeying.

The rekeying was originally something that the commercial SSH2 package did and that sometimes caused problems early on when talking to an OpenSSH clients.

That was then. This is now and it seems that OpenSSH 3.9 knows all about rekeying sessions (I think this was added in 3.6?). The OpenSSH 3.9 server I was connecting to automatically attempts rekeying with SSH-2 clients after ~1GB of data transfer and as far as I can tell there’s no configuration to turn if off. There is an poorly documented ‘RekeyLimit’ for that version that is supposed to control how much data is transfered before rekeying but it had no effect in my hands and I do not see it attempting to regulate anything in the source code (though I’m not very good at C so maybe I’m missing something).

The proper fix I suppose would be to patch Net::SSH::Perl so it implements rekeying. However, I’m not nearly well versed enough in the SSH-2 protocol to do that, so I needed to work around this issue.

After further digging around in the OpenSSH source code I discovered that the server is compiled with a list of known clients that do not support session rekeying (see client strings with SSH_BUG_NOREKEY in OpenSSH compat.c).

Net::SSH::Perl reports itself to the server as version 1.30 (which it is) as the logging from the OpenSSH server in debug mode demonstrates:

Feb 18 13:48:09 olive sshd[18746]: debug1: Client protocol version 2.0; client software version 1.30

‘1.30’ is not on OpenSSH’s compat.c list of SSH_BUG_NOREKEY clients. I could add it to the list and recompile but I don’t have admin access to the sshd server I use in production (the debug messages were from my test server). Therefore the fix has to be in the client.

So, I changed the $VERSION of Net::SSH::Perl to ‘OpenSSH_2.5.2’. The OpenSSH server recognizes it’s talking to a non-rekeying client and skips the session rekeying process. Now large transfers complete without hanging.

JRockit Liftoff

I reported yesterday being unable to use the JRockit JVM when launching Tomcat with jsvc -user tomcat. I put strace -f on the process and came up with

[pid 13655] open(“/proc/self/maps”, O_RDONLY) = -1 EACCES (Permission denied)

That wasn’t surprising, I expected a /proc read problem based on my previous research, but this nailed it. strace has help me out on a few other occasions (even though I barely understand the output). I really must remember to use it sooner in the troubleshooting process.

Anyway, I delved into the source code for jsvc (jsvc-unix.c) where I was introduced to the world of Linux capabilities. Long story short, I implemented CAP_DAC_READ_SEARCH in the relevant macros, permitting the switched user to read /proc (and any other file on the system for that matter).

Here is my patch of jsvc-unix.c (from jsvc source shipped with the Apache Tomcat 5.5.20 distribution).

— jsvc-unix.c.dist 2007-02-05 22:34:01.000000000 -0500
+++ jsvc-unix.c 2007-02-05 23:41:18.000000000 -0500
@@ -115,12 +115,15 @@
#define CAPSMAX (1 << CAP_NET_BIND_SERVICE)+ \
(1 << CAP_DAC_READ_SEARCH)+ \
(1 << CAP_DAC_OVERRIDE)
-/* That a more reasonable configuration */
+/* That a more reasonable configuration.
+ CAP_DAC_READ_SEARCH permits reading /proc/self */
#define CAPS (1 << CAP_NET_BIND_SERVICE)+ \
+ (1 << CAP_DAC_READ_SEARCH)+ \
(1 << CAP_SETUID)+ \
(1 << CAP_SETGID)
/* probably the only one Java could use */
-#define CAPSMIN (1 << CAP_NET_BIND_SERVICE)
+#define CAPSMIN (1 << CAP_NET_BIND_SERVICE)+ \
+ (1 << CAP_DAC_READ_SEARCH)
static int set_caps(int caps)
{
struct __user_cap_header_struct caphead;

With patched jsvc in service, Tomcat happily runs with JRockit under a non-privileged user.

Related:

$ man 7 capabilities
BEA JRockit

Postscript
I’m reposting the pre-patch error from catalina.err here for Google to index. Maybe this post will be discovered and help someone encountering the same problem.

05/02/2007 23:13:16 12004 jsvc.exec error: Cannot create Java VM
05/02/2007 23:13:16 12003 jsvc.exec error: Service exit with a return value of 1
[WARN ] pthread_getattr_np failed in psiGetStackInfo for 0x2a9557b960.
[WARN ] OS message: Permission denied (13)

Loading… Loading… um, maybe not.

I rarely use GMail’s web interface – I don’t like it as well as desktop mail clients. For my primary account I retrieve messages via POP3 to my desktop mail client (all the while wishing Google would get busy with the IMAP) and my other low-volume, primarily read-only accounts are monitored with RSS feeds (well, technically they are Atom feeds). However, this week I needed to email out from one of the read-only accounts and that meant a trip to GMail web. Only I couldn’t get there on my PowerBook as both Safari and Firefox would hang at the blank ‘Loading…’ screen. The stuck at Loading… scenario Googles well but the top hits were of no help. Clear cache. Check. Clear cookies. Check. Adjust security settings in ZoneAlarm. Ch… “ZoneAlarm? WTF is that”, asks my PowerBook. You don’t want to know.

So I went off and did my business in Firefox on my Ubunutu box. Problem worked around but mystery not solved and that bugged me. So I committed some mental background processes to work on the problem. I was not surprised that Safari wasn’t playing nice with GMail but for both Safari and FF to exhibit the same symptoms was unusual. I ruled out problems with my GMail accounts – they were accessible from other machines – so the focus was on the commonality of Safari and FF on my Powerbook.

Eventually I came up with ‘userContent.css’ as just such a commonality. Years ago I installed a userContent.css file to suppress ads in web pages and to override style annoyances such as underlined links. It’s been quietly doing it’s job all this time and I had all but forgotten about it. Sure enough, disabling this custom CSS permitted GMail to load normally. A binary search of the contents led me to the specific offending line:

IFRAME[SRC*="ad."] { display: none ! important; }

I removed this line and userContent.css is now GMail compatible.

The bulk of my userContent.css originated from floppymoose.com with a few of my own edits. I paid floppymoose a visit and lo ‘n behold they report userContent.css is “now GMail compatible”.

That’s what I said.

Related:

floppymoose userContent.css
GMail via Atom
GMail::IMAPD
gmailproxy
Mike Davidson and comments on GMail over IMAP