Slaptijack Title

Why Isn't tmpreaper Working?

by Scott Hebert

If you have a directory that you want to keep clean, tmpreaper is a great way to remove files based on how old they are. The other day, I had a directory that looked like this:

x@x:~/dump$ ls -l
-rw-r--r-- 1 x x 212268169 Mar 15 01:02 x-2015-03-15.sql.gz
-rw-r--r-- 1 x x 212270156 Mar 16 01:02 x-2015-03-16.sql.gz
-rw-r--r-- 1 x x 212276275 Mar 17 01:02 x-2015-03-17.sql.gz
-rw-r--r-- 1 x x 212308369 Mar 18 01:02 x-2015-03-18.sql.gz
-rw-r--r-- 1 x x 212315343 Mar 19 01:02 x-2015-03-19.sql.gz
-rw-r--r-- 1 x x 212324575 Mar 20 01:02 x-2015-03-20.sql.gz
-rw-r--r-- 1 x x 212341738 Mar 21 01:02 x-2015-03-21.sql.gz
-rw-r--r-- 1 x x 212375590 Mar 22 01:02 x-2015-03-22.sql.gz
-rw-r--r-- 1 x x 212392563 Mar 23 01:02 x-2015-03-23.sql.gz

I decided that I didn't need those SQL dumps older than 30 days, so I would use tmpreaper to clean them. The proper command is

tmpreaper +30d /home/x/dump

I added the --test flag just to make sure my command would work. But, alas! Nothing happened:

x@x:~/dump$ tmpreaper --test +30d /home/x/dump
(PID 4057) Pretending to clean up directory `/home/x/dump'.

That's when I realized that this is the directory that I had recently recompressed all the files using gzip -9 to get better compression. Although ls was reporting the time the original files were created, this is not the time tmpreaper was looking at. You can see what I mean if you use ls -lc:

x@x:~/dump$ ls -lc
total 7863644
-rw-r--r-- 1 x x 212268169 Apr 11 03:03 x-2015-03-15.sql.gz
-rw-r--r-- 1 x x 212270156 Apr 11 03:06 x-2015-03-16.sql.gz
-rw-r--r-- 1 x x 212276275 Apr 11 03:09 x-2015-03-17.sql.gz
-rw-r--r-- 1 x x 212308369 Apr 11 03:12 x-2015-03-18.sql.gz
-rw-r--r-- 1 x x 212315343 Apr 11 03:15 x-2015-03-19.sql.gz
-rw-r--r-- 1 x x 212324575 Apr 11 03:17 x-2015-03-20.sql.gz
-rw-r--r-- 1 x x 212341738 Apr 11 03:20 x-2015-03-21.sql.gz
-rw-r--r-- 1 x x 212375590 Apr 11 03:23 x-2015-03-22.sql.gz
-rw-r--r-- 1 x x 212392563 Apr 11 03:26 x-2015-03-23.sql.gz

In order to have tmpreaper look at the proper timestamp, use the --mtime flag:

x@x:~/dump$ tmpreaper --mtime --test +30d /home/x/dump
(PID 4362) Pretending to clean up directory `/home/x/dump'.
Pretending to remove file `/home/x/dump/x-2015-03-20.sql.gz'.
Pretending to remove file `/home/x/dump/x-2015-03-17.sql.gz'.
Pretending to remove file `/home/x/dump/x-2015-03-21.sql.gz'.
Pretending to remove file `/home/x/dump/x-2015-03-15.sql.gz'.
Pretending to remove file `/home/x/dump/x-2015-03-22.sql.gz'.
Pretending to remove file `/home/x/dump/x-2015-03-18.sql.gz'.
Pretending to remove file `/home/x/dump/x-2015-03-19.sql.gz'.
Pretending to remove file `/home/x/dump/x-2015-03-16.sql.gz'.

Python urllib2 with gzip or deflate encoding

by Scott Hebert

Python LogoThe other night I was working on some Python code that interacts with the zKillboard API. The API call returns kill information via JSON for EVE Online. While making the request, I was getting an error that I was using "non-acceptable encoding." It turns out that the zKillboard guys require you to accept gzip or deflate encoding in order to save on bandwidth. Here's a snippet showing how I added the "Accept-encoding" header to my urllib2 request:


import json
import urllib2
 
request = urllib2.Request('https://zkillboard.com/api/kills/solarSystemID/30000142/pastSeconds/86400/')
request.add_header('Accept-encoding', 'gzip,deflate')
 
data = urllib2.urlopen(request)
 
kills = json.load(data)

For all you EVE Online guys out there, the following API request will give you the kills in Jita over the last 24 hours. If you like the idea of a video game you can write code against, check out EVE Online's free trial. That link will give you 21 days free versus the usual 14.

Related Reading:

WARNING: The following packages cannot be authenticated!

by Scott Hebert

We run several (read: hundreds) of servers that are still running Debian 6 (Squeeze). A few months ago, we started seeing the following errors coming from the daily apt cronjob: "WARNING: The following packages cannot be authenticated!" When running apt-get update the following errors dump out:

W: GPG error: http://mirror.internode.on.net squeeze-backports Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 8B48AD6246925553
W: GPG error: http://mirror.internode.on.net squeeze-lts Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 8B48AD6246925553
W: GPG error: http://mirror.internode.on.net squeeze-updates Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 8B48AD6246925553

There are two ways to solve the problem:

apt-get install debian-archive-keyring will install all the keys you need.

If you want to install a specific key, then apt-key adv --keyserver pgpkeys.mit.edu --recv-keys 8B48AD6246925553 will do what you need. Obviously, adjust the key accordingly.

OS X Not Appending Search Domains - Yosemite Edition

by Scott Hebert

FinderIt seems this problem has resurfaced again with the new version of Mac OS X. More specifically, this problem seems to affect appending search domains when the hostname contains a dot. In Yosemite (10.10), mDNSResponder has been replaced with discoveryd. Fortunately, all we need to do here is add the --AlwaysAppendSearchDomains argument to the LaunchDaemon startup file and everything should work as expected.

  1. Before you do anything, make sure you have updated to at least OS X 10.10.1.
  2. You will need to edit /System/Library/LaunchDaemons/com.apple.discoveryd.plist. Add <string>--AlwaysAppendSearchDomains</string> to the list of strings in the ProgramArguments <array>.
  3. Restart discoveryd to see your changes take effect.
    sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.discoveryd.plist
    sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.discoveryd.plist
  4. Profit!

Mac OS X Not Using Search Domains

by Scott Hebert

FinderEvery time I restart my OS X Mountain Lion (10.9) laptop, it stops using the DNS search domains I've added via the Network preferences pane. I have found that restarting mDNSResponder fixes this issue. There are two ways to do this.

The first is a simple restart by sending a SIGHUP to the process:

$ sudo killall -HUP mDNSResponder

The other option is to stop and start the process with the launchctl command:

$ sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.mDNSResponder.plist
$ sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.mDNSResponder.plist