Server Updates (UPDATED 28 JUL 2015) OLD

[admin]

Please see new thread http://forum.freepgs.com/index.php?topic=2099.0

28 Jul 2015
PHP 5.4.43 compiled and replaced 5.4.40 on EAST, EU
PHP 5.5.27 compiled and replaced 5.5.24 on EAST, EU
LAX2 is on automatic update, cPanel servers also automatically update PHP (for selector versions)
Updated fail2ban on EAST, EU

23 Jul 2015
Updated kernel on EAST, EU, LA2, LAX2, NYC, WEST
Updated native PHP on NYC and LA2 to 5.4.43
Rebooted all servers (to be sure kernel updates are running)

10 Jul 2015
Updated kernel on EAST, EU, LA2, LAX2, NYC, WEST, WEST1
Updated openssl on EAST, EU, LA2 (auto), LAX2, NYC (auto), WEST, WEST1
Updated system PHP on EAST, EU, LAX2, WEST

31 May 2015
Removed old LAX server from service (earlier this week)
Updated BIND on EAST, EU, MTA, NYC (auto 5/21), WEST (already installed on LAX2)
Updated glibc on EAST, EU, LAX2, MTA, NYC (auto 5/29), WEST
Updated TOMCAT on EAST, EU, WEST
Updated PAM, OPENSSL on WEST1

21 May 2015
Brought new server "LAX2" online to replace the failing LAX
This machine is running the latest release of Plesk 12.0.18 with all updates.
We have also brought a machine "LA2" online that uses cPanel,/CloundLinux but we will continue to monitor LAX2 to see if we should keep it around.
PHP 5.3.29, 5.4.41, 5.5.25, 5.6.9 are available for use on the new LAX2 Plesk server.
Because the Plesk PHP packages are different from our compiled binaries on EAST and EU, additional work is necessary when migrating between these servers.  We hope to eventually upgrade all servers to the latest version of Plesk and/or expand our cPanel offering in the near future.

8 May 2015
Minor binary updates on EAST, EU, LAX, MTA, NYC, WEST, WEST1

25 Apr 2015
Kernel updates on EAST, EU, LAX, MTA, NYC, WEST
Minor binary updates on EAST, EU, LAX, MTA, NYC, WEST, WEST1
PHP 5.4.40 compiled and replaced 5.4.39 on EAST, EU, LAX
PHP 5.5.24 compiled and replaced 5.5.23 on EAST, EU, LAX

17 Apr 2015
Enabled NGINX on EAST again now that other issues are resolved
Java updates on EAST, EU, LAX, WEST  (Plesk servers with Tomcat installations)
Performance issues seem to be completely resolved on EAST

13 Apr 2015
Rekeyed the Panel and Mail SSL certificate on NYC to use SHA2
Increased number of FastCGI processes allowed from 20 to 40 on EAST
*This should correct FastCGI performance issues with the EAST server.

9 Apr 2015
Minor binary updates on EAST, EU, LAX, NYC, MTA, WEST
MySQL optimization for InnoDB tables performed on EAST

7 Apr 2015
Corrected SSL Protocols and Ciphers on NYC to achieve an A rating on ssllabs for SSL websites

4 Apr 2015
openssl and other binary updates on EAST, EU, LAX, MTA, NYC, WEST, WEST1
PHP 5.4.39 compiled and replaced 5.4.38 on EAST, EU, LAX
PHP 5.5.23 compiled and replaced 5.5.22 on EAST, EU, LAX

18 Mar 2015
PHP 5.4.38 compiled and replaced 5.4.36 on EAST, EU, LAX
PHP 5.5.22 compiled and replaced 5.5.20 on EAST, EU, LAX

17 Mar 2015
Kernel updates on EAST, EU, LAX, MTA, NYC, WEST   (no changes on WEST1)
Other minor binary updates on all servers
Updated default (system) PHP version on NYC to 5.4.38

27 Feb 2015
Minor binary updates on EAST, EU, LAX, MTA, NYC, WEST, WEST1   (NYC automatically updates)
Updated native PHP version to 5.4.38 on NYC
NYC server rebooted due to pending kernel changes previously updated but never put into place

1 Feb 2015
Updated Kernel, OpenSSL, glibc on EAST, EU, LAX, MTA, NYC, WEST
Minor binary updates on EAST, EU, LAX, MTA, NYC, WEST, WEST1

16 Jan 2015
Minor binary updates on EAST, EU, LAX, MTA, NYC, WEST

14 Jan 2015
Updated glibc on EAST, EU, LAX, MTA, NYC, WEST
Updated openssl on WEST1
Adjusted iptables rules to block more Invalid state packets on EAST
Disabled nginx reverse proxy on EAST due to DDoS attacks causing sites to become unreachable
*This will result in higher load on the server but should cause your sites to become more responsive during these attacks
*Once things are back to normal, it will be enabled again to keep from slow downloads from holding on to a connection slot.
*If things work better this way, we will keep it disabled
When time permits we will start working on a campaign to get more users to migrate from EAST to NYC or another cPanel server
We will need to start removing more of the old servers from service

25 Dec 2014
Kernel and other updates on EAST, EU, LAX, MTA, NYC, WEST, WEST1
PHP 5.4.36 compiled and replaced 5.4.35 on EAST, EU, LAX
PHP 5.5.20 compiled and replaced 5.5.19 on EAST, EU, LAX
PHP 5.4.36 compiled as native version on NYC
PHP 5.4.36, 5.5.20, 5.6.4 available on NYC via PHP Selector

13 Dec 2014
BIND and other updates on EAST, EU, LAX, MTA, NYC, WEST, WEST1
Due to more customers sites being compromised, we will be forced to remove PHP 5.2 and 5.3 from all PLESK servers.
PHP 5.4.35 compiled and replaced 5.4.34 on EAST, EU, LAX
PHP 5.5.19 compiled and replaced 5.5.18 on EAST, EU, LAX
PHP 5.2.17 and 5.3.29 removed from EAST, EU, LAX  Users switched to 5.4.35 that were using these versions.

4 Dec 2014
nss updates on EAST, EU, LAX, MTA, NYC, WEST, WEST1

2 Dec 2014
Cyrus updates on EAST, EU, LAX, MTA, NYC WEST, WEST1
(NYC updated automatically with these changes a few days ago)
(LAX previously updated)
mysqlnd update for one of the PHP versions on NYC

18 Nov 2014
kernel, openssh, perl updated on EAST, EU, LAX, MTA, NYC, WEST
Minor updates on WEST1
PHP updated to 5.4.35, 5.5.19, 5.6.3 on NYC (should be released shortly)
PHP 5.4.35 being default as "native" PHP version on NYC

7 Nov 2014
openssl, curl, java updates on EAST, EU, LAX, MTA, NYC, WEST
php (system) updates on EAST, EU, LAX, WEST (all Plesk servers)
Various binary updates on WEST1
All systems rebooted due to OpenSSL updates -- to be sure new binary is being used.
Verified PHP 5.4.34, 5.5.18, and 5.6.2 are on NYC in the PHP Selector
PHP 5.4.34 compiled and replaced 5.4.33 on EAST, EU, LAX
PHP 5.5.18 compiled and replaced 5.5.17 on EAST, EU, LAX

29 Oct 2014
CentOS Release 6.6 (from 6.5) installed on EAST, EU, LAX, MTA, WEST
CloudLinux Release/updates on NYC
Various Binary updates on EAST, EU, LAX, MTA, NYC, WEST, WEST1

16 Oct 2014
OpenSSL update on EAST, EU, LAX, MTA, NYC, WEST, WEST1
All servers rebooted to be sure new libraries are being used

14 Oct 2014
MySQL update on NYC (auto update)

9 Oct 2014
Minor binary updates on EAST, EU, LAX, MTA, NYC, WEST, WEST1
(NYC updated via automatic process on 8 Oct 2014)

1 Oct 2014
Minor binary updates on EAST, EU, LAX, MTA, NYC, WEST, WEST1
System PHP updates on all Plesk servers EAST, EU, LAX, WEST
CloudLinux Kernel update on NYC
CentOS Release update on WEST1 (5.x)

26 Sep 2014
Another BASH update on EAST, EU, LAX, MTA, NYC, WEST, WEST1
Other minor updates on EAST, EU, LAX, MTA, NYC, WEST

24 Sep 2014
BASH updated on EAST, EU, LAX, MTA, NYC, WEST, WEST1

23 Sep 2014
Minor updates on EAST and WEST physical servers.
PHP 5.4.33 and 5.5.17 made available on NYC server via automatic update on 9/22/2014
A few other minor updates also installed on NYC during this automatic update.
Issue corrected on NYC - Select PHP Version - where timezone was defaulting to UTC incorrectly.
Native PHP version updated from 5.4.32 to 5.4.33 on NYC

21 Sep 2014
Minor updates on WEST1 (DirectAdmin) server

20 Sep 2014
PHP 5.4.33 compiled and replaced 5.4.32 on EAST, EU, LAX
PHP 5.5.17 compiled and replaced 5.5.16 on EAST, EU, LAX

18 Sep 2014
Minor binary updates on EAST, EU, LAX, MTA, WEST (no changes on NYC or WEST1)
Verified that the release version of PHP 5.6 (5.6.0) is being used when PHP 5.6 is selected  (NYC - Select PHP Version)
*NYC server is set to update automatically so these logs may not reflect all updates that are being performed

15 Sep 2014
Updated kernel and other binaries on EAST, EU, LAX, MTA, WEST
Updated procmail on EAST, EU, LAX, WEST, WEST1

7 Sep 2014
Fail2Ban updated on EAST, EU, LAX, MTA, WEST
Updated Gold and Titanium plans on NYC to always include Shell access
Forum moved from EAST to NYC server
System.FreePgs site moved from EAST to NYC server
Ticket system moved from EAST to NYC server

6 Sep 2014
Updated to Apache 2.2.29 on NYC and WEST1

4 Sep 2014
Various binary updates on EAST, EU, LAX, MTA, WEST
Kernel update on WEST1

2 Sep 2014
Kernel Updates (LVE) on NYC server
Various binary updates on EAST, EU, LAX, MTA, NYC, WEST, WEST1

22 Aug 2014
PHP 5.4.32 compiled and replaced 5.4.31 on EAST, EU, LAX
PHP 5.5.16 compiled and replaced 5.5.15 on EAST, EU, LAX
Installed Fail2Ban on EAST, EU, LAX, WEST protecting SSH and SMTP AUTH so far

15 Aug 2014
PHP 5.3.29 compiled and replaced 5.3.28 on WEST1 (DirectAdmin Server)

14 Aug 2014
PHP 5.3.29 compiled and replaced 5.3.28 on EAST, EU, LAX
**This is the final version of PHP 5.3**
OpenSSL, Tomcat and other minor updates on EAST, EU, LAX, WEST
OpenSSL updated on WEST1

10 Aug 2014
Various binary updates on EAST, EU, LAX, WEST
System PHP updates [backported security updates] on EAST, EU, LAX, WEST
 (only for those using "Apache Module", FastCGI users are already using the latest versions)

3 Aug 2014
Kernel updates on EAST, EU, LAX, WEST

27 Jul 2014
Kernel updates on EAST, EU, LAX, WEST, WEST1
Apache updates on EAST, EU, LAX, WEST
Various binary updates on EAST, EU, LAX, WEST, WEST1
PHP 5.4.31 compiled and replaced 5.4.30 on EAST, EU, LAX
PHP 5.5.15 compiled and replaced 5.5.14 on EAST, EU, LAX

24 Jul 2014
More optimizations on EAST MySQL server

10 Jul 2014
Kernel updates on EAST, EU, LAX, WEST, WEST1
Various binary updates on EAST, EU, LAX, WEST, WEST1
PHP 5.4.30 complied and replaced 5.4.29 on EAST, EU, LAX
PHP 5.5.14 compiled and replaced 5.5.13 on EAST, EU, LAX

6 Jun 2014
Further MySQL optimizations were performed on EAST, EU, and LAX servers

5 Jun 2014
Various updates on EAST, EU, LAX, WEST, WEST1
RHSA-2014:0625-1 (from today) OpenSSL security update on EAST, EU, LAX, WEST
 (WEST1 not affected due to an older library being used) - RHSA-2014:0624-1 covers this version
CVE-2014-0224, CVE-2014-0221, CVE-2014-0195, CVE-2014-0198, CVE-2014-3470, and others
Restarted Apache and Nginx on EAST, EU, LAX, WEST to remove vulnerable library from memory

4 Jun 2014
Adjusted MySQL configuration on EAST, EU servers to offer greater performance

3 Jun 2014
Various updates on EAST, EU, LAX, WEST, WEST1
PHP 5.4.29 compiled and replaced 5.4.28 on EAST, EU, LAX
PHP 5.5.13 compiled and replaced 5.5.12 on EAST, EU, LAX

11 May 2014
Kernel updates on EAST, EU, LAX, WEST, WEST1

5 May 2014
Tomcat updates on EAST, EU, LAX, WEST
PHP 5.4.28 compiled and replaced 5.4.27 on EAST, EU, LAX
PHP 5.5.12 compiled and replaced 5.5.11 on EAST, EU, LAX
ionCube loader updated to 4.6.1 (from 4.6.0) on EAST, EU, LAX, WEST

22 Apr 2014
mod_perl updates on EAST, EU, LAX, WEST
LAX and WEST1 have a 2 hour downtime that will happen within the next 7 days due to servers being moved

20 Apr 2014
java, nss updates on EAST, EU, LAX, WEST
Minor updates on WEST1

11 Apr 2014
Sent mass email to all users recommending that everyone change all of their passwords to be safe.

9 Apr 2014
Rebooted EAST, EU, LAX, WEST to resolve the vulnerable OpenSSL libraries still running in memory on these servers.
Fixed Apache service so that it automatically starts on LAX (all other servers were already set this way).

8 Apr 2014
OpenSSL updates on EAST, EU, LAX, WEST

6 Apr 2014
Updated to IonCube 4.6.0 on EAST, EU, LAX
Fixed default PHP via Apache Module (5.3.3) to use latest Ioncube loaders on EAST, EU, LAX, WEST

5 Apr 2014
Updated Apache to 2.2.27 on WEST1

4 Apr 2014
Compiled PHP 5.4.27 to replace 5.4.26 on EAST, EU, LAX
Compiled PHP 5.5.11 to replace 5.5.10 on EAST, EU, LAX
Apache and OpenSSL updates on EAST, EU, LAX, WEST

3 Apr 2014
Minor updates on EAST, EU, LAX, WEST, WEST1

2 Apr 2014
EUR has been removed from service due to lack of interest

28 Mar 2014
Kernel updates on EAST, EU, EUR, LAX, WEST

22 Mar 2014
Minor library and binary updates on all servers

15 Mar 2014
Compiled PHP 5.4.26 and 5.5.10 on EAST, EU, EUR, and LAX servers
Replaced 5.4.25 with 5.4.26 and 5.5.9 with 5.5.10 on these same servers
Minor library and binary updates on all servers

1 Mar 2014
Postgresql Updates along with a few other binaries on EAST, EU, EUR, LAX, WEST
libtiff updated on EAST, EU, EUR, LAX, WEST, WEST1

20 Feb 2014
Minor updates on EAST, EU, EUR, LAX, WEST, WEST1 servers
Postfix update on WEST (It is running Qmail as the MTA on that server, Postfix was slightly updated from OS vendor)

15 Feb 2014
Fixed SSL certificate used by Courier (IMAP+POP3) and Postfix (SMTP) on EAST, EU, EUR, LAX
Added customized FTP timeout and passive port settings to EUR
Applied SSL certificate to EUR

14 Feb 2014
Enabled KAV (Antivirus) for a limited number of users on EU

13 Feb 2014
Updated kernel to 2.6.32-431 on EAST, EU, EUR, LAX, WEST
Updated mysql to 5.1.73 on EAST, EU, EUR, LAX   (MySQL already 5.5.33 on WEST)
No changes on WEST1 due to old VPS host system kernel.
Switched on server-wide greylisting support on EAST, EU, EUR, and LAX (enabled earlier on WEST)

11 Feb 2014
nss and wget updates on EAST, EU, EUR, LAX, and WEST
php-pecl-apcu update on WEST
Enabled PIPED logs on WEST
Compiled PHP 5.2.17, 5.3.28, 5.4.25, and 5.5.9 on EUR in case we decide to start using it
browscap.ini updated to version 5023 on EAST, EU, EUR, and LAX

10 Feb 2014
PIPED logs have been enabled on EAST due to too many open files causing errors for some users
A user script on EU was used to send many tens of thousands of messages.
 This confirms that the rate limits in Postfix are working correctly because messages were still being sent.
 This allowed us to stop the outbreak before hundreds of thousands of messages were sent like the other day.

9 Feb 2014
Compiled new PHP 5.4.25 on EAST, EU, LAX, replaced those with 5.4.24 to 5.4.25, removed 5.4.24
Compiled new PHP 5.5.9 on EAST, EU, LAX, replaced those with 5.5.8 to 5.5.9, removed 5.5.8
Switched on server-wide greylisting supoort on WEST.  More will follow if that functionality works correctly.
Decreased from 2 to 1 spamd worker processes on EU to reduce CPU load

8 Feb 2014
This morning, the same problem occurred, but we were able to get it online promptly.
Further limits were placed in the Postfix config files
A user was contacted that sends email to send in smaller batches in case the limits aren't stopping it.
A few more users were adjusted so the setting "Unknown Address forward to ..." was disabled.
(This causes all email to be accepted then forwarded to the external address listed which then ends up bouncing.)
Checked files with maldet (all that were modified within last 5 days) and rkhunter was run on EU
All domains were changed to the recommended option of "Reject" for unknown addresses
Disabled some PHP functions globally for users on EU to be sure a script isn't being used to generate system calls

7 Feb 2014
EU was taken down earlier, but we were able to get it turned back on within an hour.
Further changes  have been made to prevent the spamming via one possible exploit path.
The EU server is currently whitelisted at the provider so it will not automatically be disconnected.
(This will end in a few hours.)
Enabled blacklist and SPF antispam measures on all servers.  (Will not help with outbound.)

6 Feb 2014
EU server was down due to abuse for 5.5 hours
yum binary updates on EAST, EU, LAX, and WEST
We will be compiling PHP 5.5.9 (released yesterday) in the near future.
We will continue monitoring SMTP connections and the mail log for the next few hours.
New server EUR.LVCS.NET setup for testing purposes

4 Feb 2014
OpenLDAP updates on EAST, EU, LAX, and WEST

3 Feb 2014
Removed SSH access from a user account on EAST that was being used to scan remote hosts for weak passwords
The user account password was likely compromised.

30 Jan 2014
Java and a few other binaries updated on EAST, EU, LAX, WEST, and WEST1 servers.

22 Jan 2014
Bind, Python and a few other binaries updated on EAST, EU, LAX, and WEST servers

19 Jan 2014
OpenSSL updates on EAST, EU, LAX, and WEST servers
New PHP versions 5.4.24 and 5.5.8 have been compiled on EAST, EU, and LAX servers
Ioncube loader updated from 4.5.0 to 4.5.2 on EAST, EU, and LAX

4 Jan 2014
Updated kernel images to 2.6.32-431.3.1.el6.x86_64 on EAST, LAX, and WEST servers
A few other binary updates were performed on these as well as EU

24 Dec 2013
Updated mysqli and pdo_mysql default_socket paths on EAST, EU, and LAX servers

23 Dec 2013
Corrected issue with DNS service adding new zones on LAX server

20 Dec 2013
Various binaries were updated on all servers (openssl, nss) along with timezone data.

19 Dec 2013
Suspended a user on West that was running all kinds of processes (probably due to an exploited script).

18 Dec 2013
Updated to PHP 5.3.28 and a few other outdated binaries on West1 (DirectAdmin) server.

17 Dec 2013
Made changes that will always make Ioncube Loader and Zend Optimizer/Guard available for all versions of PHP 5.2/5.3/5.4
(If these were previously enabled manually for you, this will cause a 500 error, contact us to remove the custom entries for your site.)
Made the above changes to EAST, EU, LAX
Updated ioncube Loader to version 4.5.0 on EAST, EU, LAX
(This version now supports PHP 5.5.x)
EU Server disk space issue resolved -- we found a customer with 35GB of log files (error_log)
Additional MySQL performance tweaks on EU server (it is processing 500 queries per second).

16 Dec 2013
Corrected permissions of the default session.save_path on West server again
Updated kernel images on EU, EAST, LAX, and WEST servers
*As a reminder, anyone needing custom PHP versions on the WEST server should request migration to the LAX server.
New PHP 5.3.28, 5.4.23, 5.5.7 compiled on LAX, swapped all that had 5.3.27 to 5.3.28, etc.
New PHP 5.3.28, 5.4.23, 5.5.7 compiled on EAST, swapped all that had 5.3.27 to 5.3.28, etc.
New PHP 5.3.28, 5.4.23, 5.5.7 compiled on EU, swapped all that had 5.3.27 to 5.3.28, etc.
The choices for 5.3.27, 5.4.20, and 5.5.4 have been removed from EAST, EU, LAX servers

10 Dec 2013
Minor updates to samba and libjpeg-turbo

9 Dec 2013
Corrected permissions of the default session.save_path on West server
Opened additional ports on all servers to allow for Passive FTP connections
Spam Assassin updated on servers that have it installed (East and EU)

5 Dec 2013
Additional MySQL performance tweaks were performed on East
West has been updated to the latest CentOS release (6.5)
West yum repository restored to default instead of pointing to the provider's outdated mirror that didn't even have this release.

4 Dec 2013
East and EU have been updated to the latest CentOS release (6.5)
The dedicated /tmp partition has been eliminated on East (this was causing problems with the Plesk backup feature).
In the near future, we may be building updated PHP versions 5.4.22 and 5.5.6 (to supplement 5.4.20 and 5.5.4 that are currently available).