Monitor mail queue with filter

check_mailq_filter – an Icinga / Nagios postfix mail queue monitor with a filter (include or exclude mails matching a pattern).

So to exclude facebook mails from your monitor you may run:

check_mailq_filter.py -x '.*facebookmail.*'

While you’re here, have a look at pfqueue – a console tool for interactively viewing your postfix queue that’s a step up on postcat etc.

Fix ARA warnings with mysql

Getting errors like “/root/.virtualenvs/utils35/lib/python3.5/site-packages/pymysql/cursors.py:170: Warning: (1300, “Invalid utf8 character string: ‘9C1DCE'”)
result = self._query(query)”  when you run an ansible playbook with the ARA plugin?

Change the mysql python lib to cymysql

ie in $ANSIBLE_CONFIG

[ara]
database = mysql+cymysql://ara:dbpass@localhost/ara

Seems like pymysql has issues with utf-8 & python v3

Add tracking IDs to your web application

With a complex multi-tier stack with HTTP requests getting proxied it can be difficult to track a request as it goes around the system.

One thing you can do is enable mod_unique_id in apache – this creates a distinct environment variable UNIQUE_ID in the web server context for each incoming request. Simply loading the module enables it.

You can then add this via header to downstream systems (eg application servers such as php-fpm or python flask uwsgi) and return upstream so you can view it with browser DevTools with the following config:

RequestHeader set my_id %{UNIQUE_ID}e
Header set my_id %{UNIQUE_ID}e

Furthermore you can add it to your webserver logs:

LogFormat "%a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D %{UNIQUE_ID}e" combinedtime
CustomLog logs/access_log combinedtime

You can do similar in nginx with $request_id.

For a far more in-depth approach to this, look at Open Tracing.

Getting pam_ssh_agent_auth to work with Ansible

pam_ssh_agent_auth lets you use your ssh keys inside ssh-agent, that you forward in your ssh client connections to subsequently give you passwordless sudo via a destination side list of trusted public keys.

This can be used with ansible which often needs root permissions, details below:

  1. There is a good (ubuntu specific) guide to setting pam_ssh_agent_auth up here for “normal” non-Ansible tasks.
  2. Next ensure you are running ssh-agent and your key is enrolled.
  3. Also ensure you are actually forwarding your ssh-agent (eg with ForwardAgent yes in ~/.ssh/config
  4. In your ansible.cfg, add -o ForwardAgent=yes to ssh_args
  5. Also in ansible.cfg, remove -n from sudo_flags (or you can customize this on a per-host basis in your inventory file, with the ansible_sudo_flags directive.
  6. Run your playbook with sudo or become directives as necessary.

idmapd for NFSv4 doesn’t do what you think it does

After wasting a lot of time trying to remap UIDs for an NFS mount using idmapd, I came across this article.

In summary:

“The problems start when we want to access the file contents or modify a file/directory and suddenly we get ‘permission denied’ type messages. Now NFS itself doesn’t do the security authentication but delegates it down to the underlying RPC mechanism.

Ok so we move down a level and look at RPC’s security. Well AUTH_SYS on RPC is just as it has been for the last umpteen years – based on uid/gid. The problem is these uids/gids are sent over the wire (just as they always have been).
“But”, says you, “Isn’t this what the whole NFSv4 id mapping feature was meant to fix!”
Ahh – there’s the rub.
RPC hasn’t been augmented to support such mapping.
Since RPC still send’s uid/gid over the wire the whole uid<->username mapping seems pretty useless in practice for the above common scenario (if you control the entire network then uids and gids are typically the same and there is no matching problem in the first place).”

You can’t modify files if you’re using AUTH_SYS (check this with an exportfs -v, if you see sec=sys in the export options, you are) with idmapd unless you make them world writeable. You can’t enter directories unless they’re world-executable.