"Any activity becomes creative when the doer cares about doing it right, or better."
-- author John Updike
Work history: Gleim Internet Inc.
From February 2010 to November 2014, I worked on the Network Operations Center team
of Gleim Internet Inc. in Gainesville, Florida. Gleim's NOC was a small team of
2-4 people in a company of about 150, responsible for maintaining a network of
about 60 physical and virtual Debian Linux servers at three sites.
- Gleim is a global industry
leader in providing training materials for accounting and aviation
professionals, including online courses.
- Debian servers supported customer and employee websites (LAMP stack
with custom code, mediawiki, etc.), the office phone system (asterisk,
FreePBX), Windows and OS X file sharing (samba), e-mail (postfix,
dovecot, SpamAssassin, RoundCube, ClamAV, etc.), instant messaging
(OpenFire), virtual private networking (OpenVPN) and network
infrastructure/administration (CFEngine, Icinga, Cacti, BIND, OpenLDAP,
ISC dhcpd, snmpd, fail2ban).
- In 2014, the NOC converted all of Gleim’s websites from apache’s
prefork model and mod_php5 to the thread-based worker model utilizing
FastCGI and FPM for PHP, to address scalability issues. All team
members contributed significant novel solutions to numerous obstacles
that arose in the course of implementation.
- The main MySQL database replication group ran Percona Server 5.5
with a data set in the hundreds of gigabytes. Gleim's NOC was
responsible for tasks often handled by dedicated database
administrators elsewhere, including schema evaluation, optimization,
scalability design, etc.
- Core employee services ran on high-availability clusters utilizing
hardware RAID, Xen for virtualization and live migration, DRBD and
Cluster LVM for disk replication and management, and Corosync and
Pacemaker for high-availability clustering.
- Gleim had a fairly simple network of three sites using static
routing, with Debian-based routers using iptables for firewalling and
squid for web caching. When the office was multihomed, the office
router also used quagga for BGP with the upstream providers.