path: root/CHANGELOG

$Id: CHANGELOG,v 1.3 2003/12/23 19:49:55 urkle Exp $

TODO:
* Port connection portion to other DBMS?  Genericize the module?  Start with 
  PostgreSQL. (provider mechanism, and libDBI)
* does determining table name in massvirtual mode upon every request
  cause performance degradation?  If so fix.
* LogSQLRotateLogs directive with daily/monthly/weekly/etc.
* new format char: IP as bigint? ( not w/ ipV6 )
* socket-based middleman daemon with configurable conns, or connect/disconnect.
* ignore by cookie
* tools to import logs into SQL
* Directive to yes/no create ancillary tables (or just access table)
* break module into separate code files
    separate DB implimentation into sub-modules via provider mechanism
* add document building to Makefile.in
* backport patch for apache 1.3

CHANGES:
1.91: 2003-12-23
* Added checks for MySQL to autoconf
* fixed merge code to work correctly.
* Merged in 1.19b1 changes
* Hostnames are now converted to lowercase in the mass-virtual naming section.
  The loop that converts dots to underscores has been optimized as well.
* New directive LogSQLTableType allows one to specify the kind of table that
  the module makes during table creation (e.g. InnoDB, MyISAM).  Thanks to Jim
  Turner for the suggestion and patch.  If your MySQL server does not support
  the specified type, it will create a MyISAM table instead.
* Directives can now be placed in the 'main' server config and will be
  inherited by the virtual hosts.  This means a LOT less repetition: you only
  specify the item once to have it inherited, but it can still be overridden
  on a virtualhost level.  These can be specified in this manner:

		LogSQLTransferLogTable
		LogSQLTransferLogFormat
		LogSQLPreserveFile
		LogSQLNotesLogTable
		LogSQLHeadersInLogTable
		LogSQLHeadersOutLogTable
		LogSQLCookieLogTable
		LogSQLRemhostIgnore
		LogSQLRequestIgnore
		LogSQLRequestAccept
		LogSQLWhichNotes
		LogSQLWhichHeadersOut
		LogSQLWhichHeadersIn
		LogSQLWhichCookies
		LogSQLWhichCookie

1.90: 2003-12-22
* updated code to compile under apache 2.0
* rewrote and consolidate configuration handler routines
* made all functions static.
* made delayed insert configurable, instead of compile time
* moved to my autoconfigure support

1.18:
* Delayed inserts (a MySQL extension) are now available at compile-time.
  THIS IS UGLY because there seems to be a bug in the way MySQL returns
  errors on failed INSERT DELAYED queries.  See the FAQ.
* Reworked the Makefile a tad.
* Added stdlib.h to the includes, some systems need it for atoi(). Added
  string.h to the includes just to be safe.
* Reworked table creation (again).  Inserts are attempted and tables
  are made only on ER_NO_SUCH_TABLE.  This should solve all the multiple-
  creation-attempt issues as well as reduce overhead.
* safe_mysql_query reworked; it now can be counted on to preserve_entry()
  if the query doesn't succeed.  [Before, preserve_entry() was called
  separately.]
* Always reset SIGPIPE to handler before leaving safe_mysql_query().
* New directive LogSQLRequestAccept, another filter for excluding things
  from being logged.
* Alphabetized the directives in the reference section of the docs.
* New format char 'a' lets one log the request arguments, i.e. the
  part of a CGI request after the ? character.  Also put a column for
  this in the create_tables.sql file.  Thanks to Dave Wilson for this.
* Reorganized the FAQ by section.
* Renamed global variables (e.g. tcp_port -> logsql_tcpport) throughout
  the code in order to ensure against naming conflicts.  tcp_port, in
  particular, seemed to be causing problems / segfaults on certain systems.
* Renamed safe_mysql_query() to safe_sql_query().
* Now check for minimum configuration info to establish db link and
  log an error if it's insufficient.


1.17:
* Renamed the module mod_log_sql to conform to the project goal of
  database-inspecificity.
* Added capability of logging Notes field.  This is useful for folks using
  custom modules that provide loggable info in the notes, e.g. mod_gzip.
  A new directive LogSQLWhichNotes configures which notes to log to the
  notes_log table.
* Added capability of logging inbound and outbound headers. New directives
  LogSQLWhichHeadersIn and LogSQLWhichHeadersOut configure which headers to
  capture.  Headers are stored in their own table or tables.
* Fixed potential segfault in preserve file function due to silly pfclose
  placement. (Only affected user if the preserve file couldn't
  be opened.)
* Changed default socket file to /tmp/mysql.sock because that's the
  default on a compiled MySQL.
* Put back-quotes (`) around table name so that names with dashes are
  legal.
* Took away the prepend of /tmp to the preserve filename.  Now the
  user can specify whatever local path they want.  I figure that
  filesystem permissions will prevent people from doing anything
  really dumb, and people have requested this change.
* Better checking in the extract_cookie routine.  Before, it could
  segfault if a person specified "c" but didn't define MySQLWhichCookie.
* Some code reorg/renaming for clarity and to support the new direction
  of database inspecificity.  More to come.
* Simplified error messages.
* Table creation now uses safe_mysql_query and checks the result.
* Config directives used to begin with MySQL, now begin with LogSQL.
  This is for database inspecificity.  In your httpd.conf just do
  a search-and-replace.
* More robust table-creation code with error checking.  [The race condition
  that several users have reported may still exist, but this will go a long
  way toward debugging the condition.]
* Fixed bug whereby a MySQL connection was abandoned on reopen; the old
  connection is now properly closed first.
* Minor: remind user not to set createtables when massvirtual is on.
* If Apache started but MySQL was unavailable, the module would do
  everything it was supposed to (preserve entries, etc.) but not notify
  the sysadmin. Added a log message to alert sysadmin if MySQL is
  unreachable at startup.
* New config key 'I' to extract & log unique_id, provided by mod_unique_id.
  unique_id is the key that links separate tables (access, notes, etc.)
* Migrated to the Artistic License (as used by Perl).
* Moved table creation into its own callable function
* Robustifying table creation
* In make_combined_log.pl, backtick-quoted the table name and added an 
  order-by clause within the select statement.
* TCP port number for db connection is now configurable using LogSqlTCPPort
* New directive LogSQLForcePreserve sends all log entries directly to the
  preserve file and *entirely bypasses* the db.  Useful for debugging, but
  can be dangerous if you forget it's set!
* Table names are now quoted in all cases with backticks in order to permit
  names containing dashes (since many hostnames contain dashes).
* The request_method field is now created as a varchar(10) instead of (6),
  after it was pointed out to me that some methods can be longer than 6.
* New directive LogSQLMachineID sets a string identifier for the webserver
  machine.  This is useful if you have a cluster of many webservers and you
  want to differentiate between them in the logs:  you can tell which log
  entries came from each machine and thereby analyze your loadbalacing
  performance.  Activate with the TransferLogFormat character 'M'.  Much
  faster than doing some sort of wacky IP addr lookup via local_addr().
* Added an alternative way of logging cookies. If you need to associate 
  multiple cookies with each request, the new way involves using 
  LogSQLWhichCookies (note the plural) and LogSQLCookieLogTable.
* Removed LogSQLRefererIgnore references, since it was never actually
  activated anyway and the idea probably was of minimal value.


1.16:
* Moved all the user DEFINEs inside the .c file -- splitting them
  between the C and the Makefile was getting just too cumbersome.
* A new MySQLPreserveFile runtime config directive.  In the last
  version the name of the preserve-file was hardcoded and therefore
  global across all Apache virtual servers.  Now the user can configure
  this on a per-virthost basis.  It defaults to a hardcoded value
  if the user does not define it.  The module *always* prepends /tmp/
  to the user-supplied value for security reasons.
* A new MySQLSocketFile runtime config directive.  In the last
  version the name of the MySQL socket was hardcoded.  Now the user
  can configure this at Apache runtime.  However, it is a global
  setting (set once) just like the rest of the actual database info is.
  It defaults to a hardcoded value if the user does not define it.
* A new MySQLCreateTables runtime config directive.  Module can now
  create the access table on-the-fly.  Table creation takes place
  during the virtual server's first request and is flagged after that to
  avoid repetition.
* A new MySQLMassVirtualHosting runtime config directive.  This flag
  currently only activates a single feature: each virtual server gets
  its very own dynamically-determined table prefixed 'access_' with the
  server's name following.  It also implies MySQLCreateTables On, and
  obviates the need for MySQLTransferLogTable.
* escape_query (was mysql_escape_log) is now called on every item  
  rather than first checking to see if it needs to be called, which
  was probably a big waste of time.  Furthermore the routine now  
  uses a native MySQL API call to do the escaping instead of doing 
  this 'manually.'  It attempts to use the charset-respectful MySQL
  call first, but falls back on a more generic call if the MySQL
  server is unavailable (e.g. if it goes offline).
* Open preserve file with pfopen instead of regular fopen to
  take advantage of pool structure.
* As forewarned, I finally got rid of the code to support separate
  Referer and Agent logs.
* Finally brought the make process up-to-date with the way Apache
  likes modules to be done.
* MySQLWhichCookie is now configurable on a per-virtualserver basis. 
  Before it was single-shot global only.
* Reduced sleep time on a retry to 1/4 second from 1/2 second.
* Confirmed that this module will compile with -pedantic ... but not
  with -ansi. :-)
  

1.15:
* Vastly improved error reporting is a lot clearer about lost db
  connections, etc.  Some unreachable code has been corrected.
* The way that query-retries and openlink-retries are handled has 
  been tweaked and improved.
* Missed database entries are now preserved for later inclusion. This
  file is not held open but is closed after each use, so it's safe
  to delete while Apache is running.
* Now each child instantiates its MySQL link upon birth rather than
  waiting for its first request.


1.14:
* Improved the apxs instructions based on user feedback, including the
  mysql.sock define issue.
* Corrected the INSTALL example directives to the new format.
* Some improvements to 'make distro'


1.13:

* Now use ap_get_server_name() in extract_virtual_host() to fix the
  instances where mass-virtual-hosting sites were getting the wrong
  server-name written to the log.
* Now use mysql_real_connect() instead of mysql_connect().  The
  latter's use was deprecated and did not work in MySQL 4.
* There is now a DEFINE for the socket name.  This is used by the 
  mysql_real_connect() function and is relevant only if the db resides
  on localhost.  It is irrelevant if the db resides on a networked
  machine and is ignored in that case (although it still must be defined
  for the connect to work).


1.12:

* Added a mysql_close() call to get rid of those annoying MySQL
  complaints every time an httpd child process terminates. (Apache
  1.3 or later.)
* Considerable code reorganization and cleanup.  


1.11:

* Completely re-worked the cookie code.  Now the user can instruct
  mod_log_mysql which cookie (out of many available) to log.  (See
  the online directives documentation and FAQ.)
* New config capability: 'H' to log the request protocol 
  (e.g. HTTP/1.1)
* New config capability: 'm' to log the request method
  (e.g. GET, PUT, etc.)
* New config capabilities: 'z' 'q' & 'Q' to log SSL_CIPHER,
  SSL_CIPHER_USEKEYSIZE and SSL_CIPHER_ALGKEYSIZE.  These require
  openssl and glibc-devel to be installed.  (See the online
  directives documentation and FAQ.)
* Fixed a bug in make_combined_log.pl that caused it to generate
  incorrect output on single-digit days.
* Fixed make_combined_log.pl to use the data logged by 'H' and 'm'.
* Migrated all log_error calls to the newer ap_log_error call.
* Added a DEBUG define to activate certain debugging/informational
  error-log messages (for devel purposes).
* I apologize for the inconvenience this may cause:  I decided to
  rename the runtime configuration directives so that they would
  make more sense and group together with a "MySQL" prefix.  They
  are now:
	MySQLRefererLogTable
        The MySQL table that holds the referer log
	MySQLAgentLogTable 
        The MySQL table that holds the agent log
	MySQLTransferLogTable 
        The MySQL table that holds the transfer log
	MySQLTransferLogFormat 
        Instruct the module what information to log to the MySQL transfer log
	MySQLRefererIgnore 
        List of referers to ignore, accesses that match will not be logged to 
MySQL
	MySQLRequestIgnore 
        List of URIs to ignore, accesses that match will not be logged to MySQL
	MySQLRemhostIgnore 
        List of remote hosts to ignore, accesses that match will not be logged 
to MySQL
	MySQLDatabase 
        The name of the MySQL database for logging
	MySQLWhichCookie 
        The CookieName that you want logged when using the 'c' config directive
	MySQLLoginInfo 
        The MySQL host, user-id and password for logging
  You'll need to just do a search-and-replace in your httpd.conf...
  Again, I'm sorry, but it really is better this way.
* Cleaned up the summaries that get output on "httpd -L"
* Expanded the enclosed access_log.sql file to support every column type
  that mod_log_mysql provides -- delete the ones that you don't need.
* Some cleanup of the Makefile - pay attention to the settings
* Made the MySQLTransferLogFormat default "AbHhmRSsTUuv" to incorporate the
  new column types and sort the characters alphabetically for reading ease.
  
  
1.10:

* New config directive/capability: 'c' to log mod_usertrack cookies.
* Some code cleanup and commenting.
* Referer and User-Agent now set to want_orig=1 ( a very minor detail )
* Corrected mysql_escape_log to properly check for and escape strings
  with `dangerous' characters.  It appears that it was doing this 
  incorrectly before.
* Deleted log_mysql_child(), a function that was never called.


1.09:

* If the MySQL INSERT causes an error, we re-try that INSERT after
  a short 1/2-second sleep just to make sure it wasn't due to a 
  network glitch or other gremlin.
* Made the default log format: huSUsbTvRA.  This provides everything
  required to reproduce Combined Log Format data.


1.08: 

* Now log a single '-' (instead of a zero-length string) when
  User-Agent is blank. This is similar to what Apache does in its own
  logs.  (Should have caught this when I did the same thing for
  Referer.)
* Separated documentation into README, INSTALL, CHANGELOG, etc. as
  appropriate.


1.07:

* Renamed TransferIgnore directive to RequestIgnore, since that's
  really a more specific and accurate description of what that
  directive means. 
* Now log a single '-' (instead of a zero-length string) when Referer
  is blank. This is similar to what Apache does in its own logs.


1.06:

* Added 'R' and 'A' options to TransferLogMySQLFormat so that we now
  can log Referer and Agent respectively. 
* Code cleanup: all compilation warnings are now gone, even with -Wall. 
  (They were mainly "const" issues that needed straightening up.)
* Added RemhostIgnore configuration directive to permit non-logging of
  any request coming from a specific host, e.g. a local network
  machine, etc. 
* Now use the non-obsolete ap_compat.h headerfile instead of
  compat.h.  This simply gets rid of a compilation warning, nothing
  more.
* Now include a headerfile (http_log.h) that was missing.  Its
  absence was giving us this warning message: "implicit declaration of
  function `ap_log_error_old'."
* For numerics that Apache customarily logs as a "-" we now log a zero
  instead of a -1.  This seems to be more intuitive, e.g. in the
  "bytes_sent" column.
* We now have a Makefile and a full "make" process that does all
  the real work.
* New maintainer.
 

1.05:

* Removed some redundant code, after being noted by Vivek Khera that
  this code doesn't even get called with the current apache code.  It
  can be done in apache 1.3, but it works ok without it anyway.
* Added the necessary include file to make the module compile under
  Apache 1.3b6.  I haven't actually tested that it works, though.
  indent'd the code.


1.04:

* Rearranged some code to allow for a successful apache 1.3beta
  compilation.  Please note that this is *untested*, I only got it to
  compile, I haven't actually tried to run apache 1.3.


1.03:

* Changed the check for 'mysql server has gone away' to be case
  insensitive, so that it works with MySQL 3.21
* Changed the behavior so that a link isn't established until it's
  necessary (e.g., if SQL logging is used for one virtual IP, a link
  won't be opened until there's an access to that IP).


1.02:

* Managed to track down that segmentation fault that occured once, and
  fixed it.  No known bugs now exist.


1.01:

* Segmentation fault in case of certain parameters lacking fixed.
* Worked around the SIGPIPE signal that's sent in certain events from
* mysql_query(). Minor modifications