From 2d71c1472c64bc47348fc4c6337a2e903e2649b3 Mon Sep 17 00:00:00 2001 From: Edward Rudd Date: Wed, 7 Apr 2004 06:21:24 +0000 Subject: Added registration function for database drivers completely separated mysql driver added optional server announce (LogSQLAnnouce On) cleaned up internal names of Database parameters added code to allow +/- of logged cookies,headers, and notes, as well as remhostignore, request allow and request ignore. reorganized command_rec again made bigger not about deprecated commands updated documentation --- Documentation/manual.xml | 274 +++++++++++++++++++++++++---------------------- 1 file changed, 143 insertions(+), 131 deletions(-) (limited to 'Documentation') diff --git a/Documentation/manual.xml b/Documentation/manual.xml index 495b15f..d7dfc12 100644 --- a/Documentation/manual.xml +++ b/Documentation/manual.xml @@ -35,6 +35,11 @@ Edward Rudd + + 1.2 + 2004-04-? + Updated for mod_log_sql v1.97 + 1.1 2004-03-02 @@ -120,50 +125,7 @@ - Platform Specific Notes - This section is currently not applicable as the new autoconf setup should auto-detect everything for your server. The win32 compilation, however, is still on my todo list. This section needs to be overhauled so do not rely completely on any information presented in it. And anyone who compiles the mod_log_sql on these platforms, please report any issues/resolutions to compiling the module. - These installation documents assume a relatively modern GNU/Linux scenario. mod_log_sql has been ported to other platforms; following are notes on compiling the module for those platforms. - - Solaris - The nanosleep() function used in mod_log_sql relies on linking aginst the librt library. Make the following alterations before proceeding: - - - In Makefile, search for the string "-lmysqlclient -lz" and change it to read "-lmysqlclient -lz -lrt" - - - In part [step:Linking] of section [sec:Static] below, change "-lmysqlclient -lm -lz" to read "-lmysqlclient -lm -lz -lrt" - - - - - BSD - No notes are available at present, but they are desired. If you have successfully ported mod_log_sql to BSD, please contact &MaintainerContact; and help fill in this section. - - - Win32 - No notes are available at present, but they are desired. If you have successfully ported mod_log_sql to Win32, please contact &MaintainerContact; and help fill in this section. - - - OS X - mod_log_sql should compile and work out-of-the-box on this platform. Here are some notes from a user successfully running the module on OS X: - The only changes I had to make were to where I had the various libraries installed. Here are the changes I made to the head of the Makefile: APACHESOURCE = /usr/local/src/apache_1.3.27 (Wasn't sure if this was really needed or not, so I downloaded the Apache source just in case) APACHEINSTALLED = /usr/sbin APACHEHEADERS = /usr/include/httpd APXS = $(APACHEINSTALLED)/apxs MYSQLLIBRARIES = /usr/local/mysql/lib MYSQLHEADERS = /usr/local/mysql/include I'm using a binary installation of MySQL and the default apache installation on OS X Client 10.2.3, the locations of these files may vary depending on how you've installed MySQL and will almost certainly be different if you're using OS X Server. - My thanks to Tom Wiebe for being the first (to my knowlege) mod_log_sql user on OS X and for providing these notes. - - - Digital Unix - Digital Unix, like Solaris, needs to be linked against librt; see section . Here are further notes from a user successfully running the module on Digital Unix: - Instead of trying to get the module to remember where the MySQL libraries were, I instead compiled apache with the information: LDFLAGS='-rpath /isp/mysql/lib/mysql' ./configure ... Everything worked as expected after that. (The error I got without this was "/sbin/loader: Fatal Error: cannot map libmysqlclient.so" ) Digital Unix (v4.0f, at least ) appears to follow the same requirements needed by Solaris, so simply adding librt to the module made it compile without errors. As for the warnings, here's the text: mod_log_sql.c: In function `extract_request_duration': mod_log_sql.c:292: warning: long int format, different type arg (arg 4) mod_log_sql.c: In function `extract_request_timestamp': mod_log_sql.c:497: warning: long int format, different type arg (arg 4) Poking around in the code, it looks like the compiler was complaining that what time() is returning doesn't play nicely with %ld by default. I just typecast them as (long)'s and the warnings went away ( not that the module wasn't working correctly without them ). The module works very well so far in testing... hasn't dropped a single log entry yet. - My thanks to Jim Turner for permitting me to quote him here, and for being the first known user of mod_log_sql on Digital Unix - - - - Do I want a DSO or a static module - This section is no longer applicable, as the new autoconf setup only allows DSO with Apache 1.3 and 2.0. This section will be removed shortly. - You need to know the answer to this question before you proceed. The answer is pretty straightforward: what have you done in the past? If you like all your Apache modules to be dynamic, then you should keep doing that. If you're more of an old-school type and prefer to compile the modules right into apache, do that. Both methods work equally well. - FWIW, the DSO method is more modern and increasing in popularity because apxs takes care of a lot of dirty little details for you. As you'll see below, the static-module method is a little more complex. - - - Installation as an Apache DSO (preferred) + Compiling and Installing Unpack the archive into a working directory. @@ -233,20 +195,23 @@ $ cd mod_log_sql-1.9 Now compile the module with GNU make. You may have to specify gmake on some systems like FreeBSD. - $ make + $ gmake If there were no errors, you can now install the module(s). If you compiled as a non-root user you may need to switch users with su or sudo. - $ su -c "make install" + $ su -c "gmake install" Password: Now edit your Apache configuration and load the modules. If you are loading the SSL logging module, you need to make sure it is loaded after mod_ssl and mod_log_sql. + If you have previously used mod_log_sql version 1.18, the name of the module has changed from sql_log_module to log_sql_module (the first parameter to LoadModule) + If you are upgrading from any release earlier than 1.97 you need to add an extra LoadModule directive to load the database driver (ie mysql). Insert these lines to either the main httpd.conf or a file included via an include directive. LoadModule log_sql_module modules/mod_log_sql.so +LoadModule log_sql_mysql_module modules/mod_log_sql_mysql.so <IfModule mod_ssl.c> LoadModule log_sql_ssl_module moduels/mod_log_sql_ssl.so </IfModule> @@ -255,6 +220,7 @@ LoadModule log_sql_ssl_module moduels/mod_log_sql_ssl.so If you are using Apache 1.3 you may need add these lines later in the configuration. AddModule mod_log_sql.c +AddModule mod_log_sql_mysql.c <IfModule mod_ssl.c> AddModule mod_log_sql_ssl.c </IfModule> @@ -312,14 +278,13 @@ mysql> source create_tables.sql Use the MySQL database called "apachelogs" running on "dbmachine.foo.com". Use username "loguser" and password "l0gg3r" to authenticate to the database. Permit the module create tables for us. Basic Example - LogSQLLoginInfo dbmachine.foo.com loguser l0gg3r -LogSQLDatabase apachelogs + LogSQLLoginInfo mysql://loguser;l0gg3r@dbmachine.foo.com/apachelogs LogSQLCreateTables on If your database resides on localhost instead of another host, specify the MySQL server's socket file as follows: - LogSQLSocketFile /your/path/to/mysql.sock + LogSQLDBParam socketfile /your/path/to/mysql.sock If your database is listening on a port other than 3306, specify the correct TCP port as follows: - LogSQLTCPPort 1234 + LogSQLDBParam port 1234 The actual logging is set up on a virtual-host-by-host basis. So, skip down to the virtual host you want to set up. Instruct this virtual host to log entries to the table "access_log" by inserting a LogSQLTransferLogTable directive. (The LogSQLTransferLogTable directive is the minimum required to log -- other directives that you will learn about later simply tune the module's behavior.) @@ -635,18 +600,55 @@ where a.id=n.id and a.id='PPIDskBRH30AAGPtAsg'; LogSQLLoginInfo - - LogSQLLoginInfo - host - user - password + + LogSQLLoginInfo + connection URI - Example: LogSQLLoginInfo foobar.baz.com logwriter passw0rd + Example: LogSQLLoginInfo mysql://logwriter:passw0rd@foobar.baz.com/Apache_log Context: main server config - Defines the general parameters of the MySQL host to which you will be logging. "host" is the hostname or IP address of the MySQL machine, and is simply "localhost" if the database lives on the same machine as Apache. "user" is the MySQL userid (not a Unix userid!) with INSERT privileges on the table defined in LogSQLTransferLogTable. "password" is that user's password. + Defines the basic connection URI to connect to the database with. The format of the connection URI is + driver://username[:password]@hostname[:port]/database + + + driver + + The database driver to use (mysql, pgsql, etc..) + + + + username + + The database username to login with INSERT privileges on the logging table defined in LogSQLtransferLogTable. + + + + password + + The password to use for username, and can be omitted if there is no password. + + + + hostname + + The hostname or Ip address of the Database machine, ans is simple "localhost" if the database lives on the same machine as Apache. + + + + port + + Port on hostname to connect to the Database, if not specified use the default port for the database. + + + + database + + The database to connect to on the server. + + + This is defined only once in the httpd.conf file. - This directive Must be defined for logging to be enabled. Or the use of LogSQLDBParam + This directive Must be defined for logging to be enabled. @@ -658,12 +660,11 @@ where a.id=n.id and a.id='PPIDskBRH30AAGPtAsg'; parameter-name value - Example: LogSQLDBParam database loggingdb + Example: LogSQLDBParam socketfile /var/lib/mysql/mysql.socket Context: main server config - This is the new method of specifying Database connection credentials and settings. This can be used to define all Database specific options and allows for a specific database driver to define new configuration attributes without modifying the mod_log_sql core. This is the preferred configuration mechanism. + This is the new method of specifying Database connection credentials and settings. This is used to define database driver specific options. For a list of options read the documentation for each specific database driver. Each parameter-name may only be defined once. - Either this directive defining the minimum of 'database','host','user','passwd' or the use of LogSQlLoginInfo and LogSQLDatabase @@ -684,60 +685,6 @@ where a.id=n.id and a.id='PPIDskBRH30AAGPtAsg'; - - LogSQLSocketFile [Deprecated] - - - LogSQLSocketFile - filename - - Example: LogSQLSocketFile /tmp/mysql.sock - Default: (database specific) - Default (MySQL): /var/lib/mysql/mysql.sock - Context: main server config - At Apache runtime you can specify the MySQL socket file to use. Set this once in your main server config to override the default value. This value is irrelevant if your database resides on a separate machine. - mod_log_sql will automatically employ the socket for db communications if the database resides on the local host. If the db resides on a separate host the module will automatically use TCP/IP. This is a function of the MySQL API and is not user-configurable. - - This directive is deprecated in favor of LogSQLDBParam socketfile [socketfilename] - This is defined only once in the httpd.conf file. - - - - - LogSQLTCPPort [Deprecated] - - - LogSQLTCPPort - port-number - - Example: LogSQLTCPPort 3309 - Default: (database specific) - Default (MySQL): 3306 - Context: main server config - Your database may listen on a different port than the default. If so, use this directive to instruct the module which port to use. This directive only applies if the database is on a different machine connected via TCP/IP. - - This directive is deprecated in favor of LogSQLDBParam tcpport [port-number] - This is defined only once in the httpd.conf file. - - - - - LogSQLDatabase [Deprecated] - - - LogSQLDatabase - database - - Example: LogSQLDatabase loggingdb - Context: main server config - Defines the database that is used for logging. "database" must be a valid db on the MySQL host defined in LogSQLLoginInfo - - This directive is deprecated in favor of LogSQLDBParam database [database-name] - This is defined only once in the httpd.conf file. - This directive Must be defined for logging to be enabled. - - - LogSQLForcePreserve @@ -801,7 +748,7 @@ where a.id=n.id and a.id='PPIDskBRH30AAGPtAsg'; Example: LogSQLTransferLogTable access_log_table Context: virtual host - Defines which table is used for logging of Apache's transfers; this is analogous to Apache's TransferLog directive. table-name must be a valid table within the database defined in LogSQLDatabase. + Defines which table is used for logging of Apache's transfers; this is analogous to Apache's TransferLog directive. table-name must be a valid table within the database defined in the LogSQLLoginInfo connection URI. This directive is not necessary if you declare LogSQLMassVirtualHosting On, since that directive activates dynamically-named tables. If you attempt to use LogSqlTransferlogTable at the same time a warning will be logged and it will be ignored, since LogSQLMassVirtualHosting takes priority. Requires unless LogSQLMassVirtualHosting is set to On @@ -905,6 +852,20 @@ where a.id=n.id and a.id='PPIDskBRH30AAGPtAsg'; Configuring What Is logged + + LogSQLTransferLogFormat + + + LogSQLTransferLogFormat + format-string + + Example: LogSQLTransferLogFormat huSUTv + Default: AbHhmRSsTUuv + Context: virtual host + Each character in the format-string defines an attribute of the request that you wish to log. The default logs the information required to create Combined Log Format logs, plus several extras. Here is the full list of allowable keys, which sometimes resemble their Apache counterparts, but do not always: + TODO: Insert Log Format Types Table + + LogSQLRemhostIgnore @@ -915,6 +876,7 @@ where a.id=n.id and a.id='PPIDskBRH30AAGPtAsg'; Example: LogSQLRemhostIgnore localnet.com Context: virtual host Lists a series of strings that, if present in the REMOTE_HOST, will cause that request to not be logged. This directive is useful for cutting down on log clutter when you are certain that you want to ignore requests from certain hosts, such as your own internal network machines. See section for some tips for using this directive. + Each string may contain a + or - prefix in a <VirtualHost> context and will cause those strings to be added (+) or removed (-) from the global configuration. Otherwise the global is completely ignored and overridden if defined in a <VirtualHost> Each string is separated by a space, and no regular expressions or globbing are allowed. Each string is evaluated as a substring of the REMOTE_HOST using strstr(). The comparison is case sensitive. @@ -929,6 +891,7 @@ where a.id=n.id and a.id='PPIDskBRH30AAGPtAsg'; Default: if not specified, all requests are 'accepted' Context: virtual host Lists a series of strings that, if present in the URI, will permit that request to be considered for logging (depending on additional filtering by the "ignore" directives). Any request that fails to match one of the LogSQLRequestAccept entries will be discarded. + Each string may contain a + or - prefix in a <VirtualHost> context and will cause those strings to be added (+) or removed (-) from the global configuration. Otherwise the global is completely ignored and overridden if defined in a <VirtualHost> This directive is useful for cutting down on log clutter when you are certain that you only want to log certain kinds of requests, and just blanket-ignore everything else. See section for some tips for using this directive. Each string is separated by a space, and no regular expressions or globbing are allowed. Each string is evaluated as a substring of the URI using strstr(). The comparison is case sensitive. This directive is completely optional. It is more general than LogSQLRequestIgnore and is evaluated before LogSQLRequestIgnore . If this directive is not used, all requests are accepted and passed on to the other filtering directives. Therefore, only use this directive if you have a specific reason to do so. @@ -944,23 +907,10 @@ where a.id=n.id and a.id='PPIDskBRH30AAGPtAsg'; Example: LogSQLRequestIgnore root.exe cmd.exe default.ida favicon.ico Context: virtual host Lists a series of strings that, if present in the URI, will cause that request to NOT be logged. This directive is useful for cutting down on log clutter when you are certain that you want to ignore requests for certain objects. See section for some tips for using this directive. + Each string may contain a + or - prefix in a <VirtualHost> context and will cause those strings to be added (+) or removed (-) from the global configuration. Otherwise the global is completely ignored and overridden if defined in a <VirtualHost> Each string is separated by a space, and no regular expressions or globbing are allowed. Each string is evaluated as a substring of the URI using strstr(). The comparison is case sensitive. - - LogSQLTransferLogFormat - - - LogSQLTransferLogFormat - format-string - - Example: LogSQLTransferLogFormat huSUTv - Default: AbHhmRSsTUuv - Context: virtual host - Each character in the format-string defines an attribute of the request that you wish to log. The default logs the information required to create Combined Log Format logs, plus several extras. Here is the full list of allowable keys, which sometimes resemble their Apache counterparts, but do not always: - TODO: Insert Log Format Types Table - - LogSQLWhichCookie @@ -988,6 +938,7 @@ where a.id=n.id and a.id='PPIDskBRH30AAGPtAsg'; Example: logSQLWhichCookies userlogin cookie1 cookie2 Context: virtual host Defines the list of cookies you would like logged. This works in conjunction with LogSQLCookieLogTable. This directive does not require any additional characters to be added to the LogSQLTransferLogFormat string. The feature is activated simply by including this directive, upon which you will begin populating the separate cookie table with data. + Each string may contain a + or - prefix in a <VirtualHost> context and will cause those strings to be added (+) or removed (-) from the global configuration. Otherwise the global is completely ignored and overridden if defined in a <VirtualHost> The table must be created (see create-tables.sql, included in the package), or LogSQLCreateTables must be set to 'On'. @@ -1003,6 +954,7 @@ where a.id=n.id and a.id='PPIDskBRH30AAGPtAsg'; Example: LogSQLWhichHeadersIn UserAgent Accept-Encodeing Host Context: virtual host Defines the list of inbound headers you would like logged. This works in conjunction with LogSQLHeadersInLogTable. This directive does not require any additional characters to be added to the LogSQLTransferLogFormat string. The feature is activated simply by including this directive, upon which you will begin populating the separate inbound-headers table with data. + Each string may contain a + or - prefix in a <VirtualHost> context and will cause those strings to be added (+) or removed (-) from the global configuration. Otherwise the global is completely ignored and overridden if defined in a <VirtualHost> The table must be created (see create-tables.sql, included in the package), or LogSQLCreateTables must be set to 'On'. @@ -1018,6 +970,7 @@ where a.id=n.id and a.id='PPIDskBRH30AAGPtAsg'; Example: LogSQLWhichHeadersOut Expires Content-Type Cache-Control Context: virtual host Defines the list of outbound headers you would like logged. This works in conjunction with LogSQLHeadersOutLogTable. This directive does not require any additional characters to be added to the LogSQLTransferLogFormat string. The feature is activated simply by including this directive, upon which you will begin populating the separate outbound-headers table with data. + Each string may contain a + or - prefix in a <VirtualHost> context and will cause those strings to be added (+) or removed (-) from the global configuration. Otherwise the global is completely ignored and overridden if defined in a <VirtualHost> The table must be created (see create-tables.sql, included in the package), or LogSQLCreateTables must be set to 'On'. @@ -1033,6 +986,7 @@ where a.id=n.id and a.id='PPIDskBRH30AAGPtAsg'; Example: LogSQLWhichNotes mod_gzip_result mod_gzip_ompression_ratio Context: virtual host Defines the list of notes you would like logged. This works in conjunction with LogSQLNotesLogTable. This directive does not require any additional characters to be added to the LogSQLTransferLogFormat string. The feature is activated simply by including this directive, upon which you will begin populating the separate notes table with data. + Each string may contain a + or - prefix in a <VirtualHost> context and will cause those strings to be added (+) or removed (-) from the global configuration. Otherwise the global is completely ignored and overridden if defined in a <VirtualHost> The table must be created (see create-tables.sql, included in the package), or LogSQLCreateTables must be set to 'On'. @@ -1040,6 +994,64 @@ where a.id=n.id and a.id='PPIDskBRH30AAGPtAsg'; + + Deprecated Command + + + LogSQLSocketFile [Deprecated] + + + LogSQLSocketFile + filename + + Example: LogSQLSocketFile /tmp/mysql.sock + Default: (database specific) + Default (MySQL): /var/lib/mysql/mysql.sock + Context: main server config + At Apache runtime you can specify the MySQL socket file to use. Set this once in your main server config to override the default value. This value is irrelevant if your database resides on a separate machine. + mod_log_sql will automatically employ the socket for db communications if the database resides on the local host. If the db resides on a separate host the module will automatically use TCP/IP. This is a function of the MySQL API and is not user-configurable. + + This directive is deprecated in favor of LogSQLDBParam socketfile [socketfilename] + This is defined only once in the httpd.conf file. + + + + + LogSQLTCPPort [Deprecated] + + + LogSQLTCPPort + port-number + + Example: LogSQLTCPPort 3309 + Default: (database specific) + Default (MySQL): 3306 + Context: main server config + Your database may listen on a different port than the default. If so, use this directive to instruct the module which port to use. This directive only applies if the database is on a different machine connected via TCP/IP. + + This directive is deprecated in favor of LogSQLDBParam tcpport [port-number] + This is defined only once in the httpd.conf file. + + + + + LogSQLDatabase [Deprecated] + + + LogSQLDatabase + database + + Example: LogSQLDatabase loggingdb + Context: main server config + Defines the database that is used for logging. "database" must be a valid db on the MySQL host defined in LogSQLLoginInfo + + This directive is deprecated in favor of the URI form of LogSQLLoginInfo. + This is defined only once in the httpd.conf file. + + + + + @@ -1314,7 +1326,7 @@ ErrorLog /var/log/httpd/server-messages Why do I get the message "insufficient configuration info to establish database link" in my Apache error log? - At a minimum, LogSQLDatabase and LogSQLLoginInfo and either LogSQLTableName or LogSQLMassVirtualHosting must be defined in order for the module to be able to establish a database link. If these are not defined or are incomplete you will receive this error message. + At a minimum, LogSQLLoginInfo in the URl form and either LogSQLTableName or LogSQLMassVirtualHosting must be defined in order for the module to be able to establish a database link. If these are not defined or are incomplete you will receive this error message. -- cgit