Release Notes for MSQL.3.4-X ============================ This is the Quadrics release of mSQL for RMS. This product contains the following rpms: qsmsqllibs contains the mSQL library and is installed on every node qsmsql contains the mSQL binary and is required on the rmshost node only Upgrade from mSQL.2.0.11 to mSQL.3.4 ------------------------------------ Databases created with mSQL.2.0.11 are incompatible with mSQL.3.4 and need to be converted. The installation scripts will backup any databases found in /var/rms/msqldb before uninstalling mSQL.2.0.11 and then restore them (to /var/rms/msqldb-3) on installing mSQL.3.4. Known Problems -------------- Upgrading from MSQL.2.0.11-2 or earlier --------------------------------------- The change to using rpm -U in the qsinstall script has resulted in a problem when upgrading the MSQL product. Because rpm executes the uninstall of the previous release after the install of the new one the preuninstall script of MSQL.2.0.11-2 is executed last and this script shuts down the msql server and also executes /sbin/chkconfig --del msqld. This effectively turns off the msqld service so that msql will not be started on reboot and also causes warning messages from the qsinstall script when it tries to update the RMS database about installed software. For example: [root@clust0 MSQL.2.0.11-3]# ./qsinstall qsinstall: Installing rpms on node clust0 (rmshost) ------------------------------------------------------------ qsinstall: clust0 : installing root rpm qsmsqllib qsmsqllib: editing /etc/services qsinstall: clust0 : installing opt rpm qsmsql qsmsql postinstall: starting mSQL server qsmsql preuninstall: shutting down mSQL server qsinstall: clust0 : WARNING : /usr/bin/rmsquery DELETE from software_products WHERE name='MSQL' AND version='2.0.11-3' FAILED : output qsinstall: clust0 : all done [root@clust0 MSQL.2.0.11-3]# This problem has been fixed in MSQL.2.0.11-3 so future upgrades will work correctly but when installing MSQL.2.0.11-3 it is necessary to uninstall any previously installed version of MSQL and then install MSQL.2.0.11-3. To do this, change directory to MSQL.2.0.11-3 and execute './qsinstall -r' to remove the current installation and then execute './qsinstall'. ======================================================= Release Notes for MSQL.3.4-39 Mar 19th 2008 ======================================================= * GNAT 9742: Fixed denial in select queries involving long text fields * Allow memory cache instrumentation * Fix off-by-one array bounds violation in parser * Don't close stdin when running in foreground with -d flag, making debugging easier ======================================================= Release Notes for MSQL.3.4-38 Feb 22nd 2008 ======================================================= * GNAT 9640: Added code to mSQL3d to detect and heal a particular type of corruption in table indexes. * GNAT 9689: Prevent long stalls caused by fsync(). * GNAT 9710: Backport fix for overflow file handling bug. ======================================================= Release Notes for MSQL.3.4-36 Aug 20th 2007 ======================================================= * Corrected miniSQL client API to correctly close database server socket in all error paths ======================================================= Release Notes for MSQL.3.4-35 Aug 14th 2007 ======================================================= * Introduced MSQL_TIMEOUT environment variable to adjust the server timeout. * Changed default server timeout from 10s to 90s to make server more robust against network congestion episodes. ======================================================= Release Notes for MSQL.3.4-34 Jun 22nd 2007 ======================================================= * Made client library signal safe. ======================================================= Release Notes for MSQL.3.4-33 Jun 15th 2007 ======================================================= * Introduced MSQL_TIMEOUT environment variable to adjust the client timeout. * Changed default client timeout from 30s to 120s to reduce chances of query failing during times of heavy I/O and large queries. * Log peer information when a connection has been dropped. ======================================================= Release Notes for MSQL.3.4-32 May 29th 2007 ======================================================= * GNAT 8910: Prevent wild queries or space starvation causing the database server to shutdown; correct out-of-space handling at server and enhance client error reporting. * Installation: Where unable to detect distribution type, fallback to Redhat compatibility ======================================================= Release Notes for MSQL.3.4-31 May 21st 2007 ======================================================= * GNAT 9018: handle loops in the AVL tree lookup code, resulting from database corruption. ======================================================= Release Notes for MSQL.3.4-30 Jun 15th 2006 ======================================================= * GNAT 8303: allow client to correctly handle a signal while communicating with the database server, by restarting poll() when it is interrupted. ======================================================= Release Notes for MSQL.3.4-29 May 30th 2006 ======================================================= * changed client library to default to ignoring SIGPIPE signals, as the previous SIGPIPE handler could cause stale database state in RMS. ======================================================= Release Notes for MSQL.3.4-28 Apr 4th 2006 ======================================================= * log daemon startup and shutdown in syslog * ensure that fd limit isn't lowered * prevent unneeded init script syslog messages ======================================================= Release Notes for MSQL.3.4-27 Mar 30th 2006 ======================================================= * added missing dependencies to ensure correct installation when installed at OS install-time. * ensure clients blocked in a read() call timeout after 30s ======================================================= Release Notes for MSQL.3.4-26 Jan 31st 2006 ======================================================= * added flush command to database protocol and API, to allow clients to ensure data writeout is started * added end-of-log marker for clarity in the log files * migrated RPM post-install rmshost checks into init script, so the 'rmshost' DNS/NIS entry doesn't need to be present at package install-time, only at service start time. * changed behaviour of mSQLd to not automatically start on a fresh install (it will be automatically restarted on upgrade, as before) ======================================================= Release Notes for MSQL.3.4-25 Sep 6th 2005 ======================================================= * The qsmsqllib RPM now automatically creates the rms group and user where not already present, and adds root to the rms group, preventing the need to run rmssetup previously. ======================================================= Release Notes for MSQL.3.4-24 Jul 27th 2005 ======================================================= * The msqld init script uses the recommended way of launching daemons, providing compatibility with Rocks 3.3 and 4.0. ======================================================= Release Notes for MSQL.3.4-23 Apr 19th 2005 ======================================================= * Improved logging of errors where mmap() calls can fail, due to tables being too big (eg >500MB on IA32) to map in the msql3d's virtual address space (GNATS PR 7389). * Added msqlfsck tool to consistency-check tables for a given database. msqlfsck checks the specified table in the given database, or where the table isn't specified, it will check all enumerated tables. Where no errors are found, the return code is 0, and 1 with errors present. At this time, repairing corruption is not implemented; the recommended procedure is to drop the table, then recreate with 'rmstblasm -u'. usage: msqlfsck [-fhv] [-d ] [-t ] -d check specific database -f fix-up table for data recovery -h show help -t check specific table -v list rows in table Eg. to check all tables in the 'rms_quad' database: # msqlfsck -d rms_quad Eg. to check the 'events' table in the 'rms_quad' database: # msqlfsck -d quad -t events The return code reflects if attention is required, so it is suitable for adding into a site-specific startup script on the rmshost. * Put rmshost checking back into the msqld init script. This had been erroneously removed when adding the LSB compliance. * Moved the 'msqladmin' binary to /usr/sbin/. ======================================================= Release Notes for MSQL.3.4-21 Dec 1st 2004 ======================================================= . Added Linux Standards Base [www.linuxbase.org] init script compliance to the msqld init script and the qsmsql postinstall and preuninstall scripts. This allows operation on any LSB-compliant Linux distribution. This will only affect non-RedHat and non-SuSE systems, for which there is native support. ======================================================= Release Notes for MSQL.3.4-20 Nov 5th 2004 ======================================================= . Imported various minor fixes from miniSQL-3.5 tree. . Fixed verification of qsmsql RPM. . Corrected error logging of most common error message to be consistent and have timestamps. ======================================================= Release Notes for MSQL.3.4-18 Sep 8th 2004 ======================================================= No functional change - just a rebuild on SLES 8 for x86_64 to ensure backwards compatibility. ======================================================= Release Notes for MSQL.3.4-17 Sep 8th 2004 ======================================================= . Fixed a bug introduced into 3.4-16 where the /var/rms/msqldb-3 hierarchy was not being created or having its permissions/ownerships changed correctly (GNAT 6975). This was causing problems for users installing from scratch. . Added checking for user 'rms' into preinstall scripts of qsmsql and qsmsqllib RPMs (GNAT 6958). ======================================================= Release Notes for MSQL.3.4-16 Aug 19th 2004 ======================================================= . Fixed a crash provoked by running the nessus port scanner against port 1114. . Added SQL function argument count checking, preventing potential crashing (GNAT 6925). ======================================================= Release Notes for MSQL.3.4-15 Apr 26th 2004 ======================================================= . Backed out previous change (to fix GNAT 6254) that was causing a memory leak (GNAT 6497); fixed GNAT 6254 correctly by copying SQL field structure for later use and free, for type TEXT fields (GNAT 6497). ======================================================= Release Notes for MSQL.3.4-14 Mar 3rd 2004 ======================================================= . Fixed crash with certain SQL queries that have string comparisons (GNAT 6254). ======================================================= Release Notes for MSQL.3.4-13 Mar 1st 2004 ======================================================= . Re-enabled ACL code that was disabled in stock non-broker variant of miniSQL 3.4 (GNAT 6242). . Moved msql3d into /usr/sbin (Linux). . Fixed reporting of what user mSQLd runs as (Linux). ======================================================= Release Notes for MSQL.3.4-12 Jan 30th 2004 ======================================================= . Corrected mSQL init script locking, so the the daemon is stopped before networking is brought down (Linux). ======================================================= Release Notes for MSQL.3.4-11 Jan 8th 2004 ======================================================= . Added fsync(2) calls to database tables; previously msync(2) was used to commit the data in the file mapping. This should increase the resilience of mSQLd in light of crashing, the file metadata will get committed as will calls to write(2). . Cleanup logging of init script (GNAT 5705, Linux). ======================================================= Release Notes for MSQL.3.4-10 Nov 23rd 2003 ======================================================= . Incorporated changes, requested by HP, to AlphaServer SC msqld startup script to prevent msql3d core dumping when starting up in single user mode. . Corrected msqld service dependencies on network (Linux/x86_64). ======================================================= Release Notes for MSQL.3.4-9 Oct 23rd 2003 ======================================================= . Added cleaning up of stale temporary tables in the Linux qsmsql postinstall and preuninstall scripts. . Changed initialisation of server daemon to change to user 'rms' in the Linux init script, rather than daemon to work around a kernel issue introduced by the ptrace patch which prevented debugging of setuid processes. . Small improvements to data syncing to minimise database corruption over an rmshost crash. . Improved table-indexing code to detect limited corruption in the database index, print an error and exit, rather than crash or spin (GNAT 5740). ======================================================= Release Notes for MSQL.3.4-8 Sept 17th 2003 ======================================================= . Changed mSQLd to use quick-sort up to certain result set sizes (default: 30K rows), this fixed a O(100) performance regression from mSQLd 2, on workloads involving two 'order by' clauses. . Logfile rotation support has been fixed to avoid mmap(2) failures, triggered by rotating log files with sustained heavy activity (Linux, GNAT 5627). This fixed the server exiting when mmap(2) failed. . Run-time control of the table-cache size has been added with the '-c' argument to msql3d and table-cache stats are available from the 'msqladmin stats' command. . The database migration script would fail to migrate databases when running with RMS-2.80. This is now fixed (Linux). . A race condition where the table-syncing code can be called when a table is being changed has been closed. This has been observed to cause rare crashes, due to the small race window (GNAT 5552). . Added connection abort handling, so the database server doesn't continue to process an SQL SELECT query when a client has aborted the connection; this prevents large run-away queries starving the system when aborted (as has been observed in logs). . Fixed mis-reporting of free space in the filesystem the database sits on when there was above ~ 10GB free space. This was causing the database server to think there was less than 20MB, at which point it was exiting. ======================================================= Release Notes for MSQL.3.4-7 July 22nd 2003 ======================================================= . Fixed crash where rows containing TEXT fields were deleted from the database and certain other conditions were met (GNAT 5453). ======================================================= Release Notes for MSQL.3.4-6 June 13th 2003 ====================================================== . Support for SuSE distribution and x86_64 added (changes to init script). . Correctly enabled corefile creation on failure. . Fixed bug where the storage file for variable-length text fields (table.ofl) would grow as records were not being recycled. . Changed msqladmin to pass a process name when connecting to msqld, so 'msqladmin stats' doesn't show '(unknown)' for the msqladmin connection. . Improved logging and fixed the case where the wrong fd could be closed when rotating logs (nightly by default). ======================================================= Release Notes for MSQL.3.4-5 May 21st 2003 ======================================================= * Changed the way mSQL defines that SQL clients access the mSQL library's error string; this allows total backward compatibility between the client and mSQL 2. * Improved the database migration (mSQL 2->3 and 3->2) code to operate in a much more rugged way, and fixed some issues. * Removed the creation of symbolic links from /usr/lib/rms to /var/rms/adm and /var/rms/msqldb-3 in the qsmsql preinstall script as they are not used and caused a problem (GNAT 5179). * Changed the qsmsql preinstall script to check that /var/rms/msqldb-3 exists before running 'find' on it to check for space required to upgrade as the script failed if it didn't exist. ======================================================= Release Notes for MSQL.3.4-4 April 11th 2003 ======================================================= * Added code to synchronise the pagecache with all memory mappings held for dirty tables, making mSQLd much more resilient to crashes. * Added additional debugging in the case where mSQLd finds a corrupt free list pointer. * Enabled connection keep-alive with appropriate paramaters to detect and remove 'dead' (ie from crashed nodes) connections. ======================================================= Release Notes for MSQL.3.4-3 (MSQL34X03 - OSF1 version only) 19th March 2003 ============================ Initial Quadrics release of mSQL 3.4. All Quadrics patches to mSQL 2.0.11 have been applied as well as the following changes: * Added hostname resolution deferral - so hostnames are looked up only when a 'msqladmin stats' command is issued. This reduces the load on the local name cache daemon to allow it to work efficiently and dramatically reduces work done in the connect hot path. * Changed mSQLd to allocate the connection array on the heap rather than the stack, supporting very large (> 100K) connections. * Reworked main mSQLd event loop and all relevant parts of the networking subsystem to use poll(2) rather than select(2) to overcome the 1024 fd limitation, including cleaner handling; general performance increases have been observed at around 5-10% and mSQLd has been tested successfully with a total of 60,000 concurrent connections. * Fixes have been made to correct subtle memory-related problems reported with Valgrind, including 'unaligned-access' OSF/1 issues. * Additional syscall checks have been added to log failed syscalls and backoff gracefully where possible. * Additional metric has been added for 'msqladmin stats' to allow the observing of the peak number of concurrent connections, to monitor how near configuration limitations the system is running. * Added split-phase connection handshaking, so as to not block the server while the server is waiting for the client to respond. * Added query sampling and logging, for profiling and gathering purposes. * Increased stride size when expanding tables, and therefore munmap(2) and mmap(2) activity, reducing system load. * Fixed intermittent server crash when inserting into tables.