diff options
Diffstat (limited to 'README')
| -rw-r--r-- | README | 151 |
1 files changed, 32 insertions, 119 deletions
| @@ -1,4 +1,4 @@ | |||
| 1 | $Id: README,v 1.4 2002/04/08 07:06:20 helios Exp $ | 1 | $Id: README,v 1.5 2002/04/21 23:01:53 helios Exp $ |
| 2 | 2 | ||
| 3 | 3 | ||
| 4 | Homepage | 4 | Homepage |
| @@ -13,35 +13,25 @@ Approach | |||
| 13 | In order to save speed and overhead, links are kept alive in between | 13 | In order to save speed and overhead, links are kept alive in between |
| 14 | queries. This module uses one SQL link per httpd process. Among other | 14 | queries. This module uses one SQL link per httpd process. Among other |
| 15 | things, this means that this module supports logging into only one | 15 | things, this means that this module supports logging into only one |
| 16 | MySQL server, and for now, also, only one SQL database (although the | 16 | MySQL server, and for now, also, only one SQL database. |
| 17 | latter limitation can be relatively easily removed). | ||
| 18 | |||
| 19 | Different data can be sent to different tables. i.e., it's possible to | ||
| 20 | define one table for TransferLog, one for RefererLog, and a 3rd for | ||
| 21 | AgentLog. [ Note: this is now deprecated behavior. Please consider | ||
| 22 | logging Agent and Referer to the same table as your transfers. ] | ||
| 23 | 17 | ||
| 24 | Virtual hosts are supported in the same manner they are in the regular | 18 | Virtual hosts are supported in the same manner they are in the regular |
| 25 | logging modules. If you specify a different table for a virtual | 19 | logging modules. If you specify a different table for a virtual |
| 26 | host it will be used, otherwise the 'general' would be used. Note: | 20 | host it will be used, otherwise the 'general' would be used. |
| 27 | since all 3 types of logs are implemented within the same module, if | 21 | |
| 28 | you specify an overriding table for a virtual host for one type of log, | 22 | SQL links are opened by each child process when it is born. Error reporting |
| 29 | it'll ignore any previous 'general' defaults (see the example in the | 23 | is robust throughout and will let you know about database issues |
| 30 | end). | 24 | in the standard Apache error-log for the server or virtual server. |
| 31 | 25 | ||
| 32 | SQL links are opened on demand (i.e., the first time each httpd needs | 26 | A robust "preserve" capability has now been implemented as well. This |
| 33 | to log something to SQL, the link is opened). In case the SQL server | 27 | permits the module to preserve any failed INSERT commands to a local |
| 34 | is down when trying to connect to it, the module remains silent and | 28 | file on its machine. In any situation that the database is unavailable -- |
| 35 | logs no error (I didn't want thousands of error messages in the | 29 | e.g. the network fails, you reboot the machine, etc. -- mod_log_mysql |
| 36 | logfile). In case the SQL link is broken ("mysql server has gone | 30 | will note this in the error log and begin appending its log entries to |
| 37 | away") a proper error message is kept to the error log (textual :), and | 31 | the preserve file. At the time that your MySQL server returns to service, |
| 38 | the module tries to reestablish the concact (and reports whether it | 32 | each of these preserve files is easily imported because it is stored in SQL: |
| 39 | succeeded or not in the error log). If the link cannot be | 33 | |
| 40 | reestablished, the module will, again, remain silent. Technical note: | 34 | # mysql -uadminuser -p mydbname < /tmp/mysql-preserve |
| 41 | The SQL link is registered using apache's pool mechanism, so SQL links | ||
| 42 | are properly closed on any normal shutdown, kill -HUP or kill -TERM. | ||
| 43 | This also means that if you restart the MySQL daemon for any reason you | ||
| 44 | should restart Apache. | ||
| 45 | 35 | ||
| 46 | 36 | ||
| 47 | 37 | ||
| @@ -63,85 +53,22 @@ What gets logged by default? | |||
| 63 | 53 | ||
| 64 | All the data that would be contained in the "Combined Log Format" | 54 | All the data that would be contained in the "Combined Log Format" |
| 65 | is logged by default, plus a little extra. Your best bet is to | 55 | is logged by default, plus a little extra. Your best bet is to |
| 66 | accept this default and employ the enclosed access_log.sql to | 56 | begin by accepting this default, then later customize the log |
| 67 | format your table. Customize your logging format after you've | 57 | configuration based on your needs. |
| 68 | had a chance to experiment with the default first. | 58 | |
| 69 | 59 | The online documentation of the run-time directives includes a full | |
| 70 | If you just want to log enough data to be able to reconstruct | 60 | explanation of what you can log, including examples. |
| 71 | a Combined Log Format log, log these: | ||
| 72 | |||
| 73 | +------------------+------------------+ | ||
| 74 | | Field | Type | | ||
| 75 | +------------------+------------------+ | ||
| 76 | | remote_host | varchar(50) | | ||
| 77 | | remote_user | varchar(50) | | ||
| 78 | | request_uri | varchar(50) | | ||
| 79 | | virtual_host | varchar(50) | | ||
| 80 | | time_stamp | int(10) unsigned | | ||
| 81 | | status | smallint(6) | | ||
| 82 | | bytes_sent | int(11) | | ||
| 83 | | referer | varchar(255) | | ||
| 84 | | agent | varchar(255) | | ||
| 85 | | request_method | varchar(6) | | ||
| 86 | | request_protocol | varchar(10) | | ||
| 87 | +------------------+------------------+ | ||
| 88 | |||
| 89 | remote_host: corresponds to the Apache %h directive. Contains the remote | ||
| 90 | hostname or IP of the machine accessing your server. | ||
| 91 | Example: si4002.inktomi.com | ||
| 92 | |||
| 93 | remote_user: corresponds to the Apache %u directive. Contains the | ||
| 94 | userid of people who have authenticated to your server, if applicable. | ||
| 95 | Example: freddy | ||
| 96 | |||
| 97 | request_uri: corresponds to the Apache %U directive. Contains the | ||
| 98 | URL path requested, excluding any query string. This is different than | ||
| 99 | the %r information you might be used to seeing: | ||
| 100 | |||
| 101 | %r: GET /cgi-bin/neomail.pl?sessionid=freddy-session-0.742143231719&sort=date_rev HTTP/1.1 | ||
| 102 | %U: /cgi-bin/neomail.pl | ||
| 103 | |||
| 104 | We log %U because it contains the real meat of the information that is | ||
| 105 | needed for log analysis, and saves the database a LOT of wasted growth | ||
| 106 | on unneeded bytes. | ||
| 107 | |||
| 108 | virtual_host: contains the VirtualHost that is making the log entry. This | ||
| 109 | allows you to log multiple VirtualHosts to a single MySQL database and | ||
| 110 | yet still be able to extract them for separate analysis. | ||
| 111 | Example: www.grubbybaby.com | ||
| 112 | |||
| 113 | time_stamp: contains the time that the request was logged. Please see | ||
| 114 | "Notes" below to get a better understanding of this. | ||
| 115 | Example: 1014249231 | ||
| 116 | |||
| 117 | status: corresponds to the Apache %t directive. Contains the HTTP status | ||
| 118 | of the request. | ||
| 119 | Example: 404 | ||
| 120 | |||
| 121 | bytes_sent: corresponds to the Apache %b directive. Contains the number | ||
| 122 | of bytes sent to service the request. | ||
| 123 | Example: 23123 | ||
| 124 | |||
| 125 | referer: corresponds to the Apache "%{Referer}i" directive. Contains the | ||
| 126 | referring HTML page's URL, if applicable. | ||
| 127 | Example: http://www.foobar.com/links.html | ||
| 128 | |||
| 129 | agent: corresponds to the Apache "%{User-Agent}" directive. Contains the | ||
| 130 | broswer type (user agent) of the software that made the request. | ||
| 131 | Example: Mozilla/3.0 (Slurp/si; slurp@inktomi.com; http://www.inktomi.com/slurp.html) | ||
| 132 | |||
| 133 | request_method: corresponds to the Apache %m directive. Contains the type | ||
| 134 | of request sent: GET, PUT, etc. | ||
| 135 | Example: GET | ||
| 136 | |||
| 137 | request_protocol: corresponds to the Apache %H directive. Contains the HTTP | ||
| 138 | protocol that was used. | ||
| 139 | Example: HTTP/1.1 | ||
| 140 | 61 | ||
| 141 | 62 | ||
| 142 | Notes | 63 | Notes |
| 143 | ----- | 64 | ----- |
| 144 | 65 | ||
| 66 | * You will customarily set most of your run-time configuration directives | ||
| 67 | on a per-virtualserver basis, with only MySQLMassVirtualHosting, | ||
| 68 | MySQLLoginInfo and MySQLDatabase 'outside' in the main server config. | ||
| 69 | Any directives other than those in the main config do NOT get inherited | ||
| 70 | by the virutal servers. | ||
| 71 | |||
| 145 | * The 'time_stamp' field is stored in an UNSIGNED INTEGER column, in the | 72 | * The 'time_stamp' field is stored in an UNSIGNED INTEGER column, in the |
| 146 | standard unix "seconds since 1/1/1970 12:00:00" format. This is | 73 | standard unix "seconds since 1/1/1970 12:00:00" format. This is |
| 147 | superior to storing the access time as a string due to size | 74 | superior to storing the access time as a string due to size |
| @@ -161,20 +88,15 @@ Notes | |||
| 161 | Log Format compliant. You can then feed this to your favorite web | 88 | Log Format compliant. You can then feed this to your favorite web |
| 162 | log analysis tool. | 89 | log analysis tool. |
| 163 | 90 | ||
| 164 | |||
| 165 | * The table's string values can be CHAR or VARCHAR, at a length of your choice. | 91 | * The table's string values can be CHAR or VARCHAR, at a length of your choice. |
| 166 | VARCHAR is superior because it truncates long strings; CHAR types are | 92 | VARCHAR is superior because it truncates long strings; CHAR types are |
| 167 | fixed-length and will be padded with spaces. Just like the | 93 | fixed-length and will be padded with spaces. Just like the |
| 168 | time_stamp described above, that kind of space waste will add up over | 94 | time_stamp described above, that kind of space waste will add up over |
| 169 | thousands of records. | 95 | thousands of records. |
| 170 | 96 | ||
| 171 | 97 | * Be careful not to go overboard setting fields to NOT NULL. If a field is | |
| 172 | * Most fields should probably be set to NOT NULL. The only ones that | 98 | marked NOT NULL then it must contain data in the INSERT or the INSERT |
| 173 | shouldn't are extra fields that you don't intend the logging module | 99 | will fail. |
| 174 | to update. (You can have other fields in the logging tables if you'd | ||
| 175 | like, but if they're set to NOT NULL then the logging module won't be | ||
| 176 | able to insert rows to these tables.) | ||
| 177 | |||
| 178 | 100 | ||
| 179 | * Apache normally logs numeric fields with a '-' character to mean "not | 101 | * Apache normally logs numeric fields with a '-' character to mean "not |
| 180 | applicable," e.g. bytes_sent on a request with a 304 response code. | 102 | applicable," e.g. bytes_sent on a request with a 304 response code. |
| @@ -183,15 +105,6 @@ Notes | |||
| 183 | makes perfect sense anyway. | 105 | makes perfect sense anyway. |
| 184 | 106 | ||
| 185 | 107 | ||
| 186 | * If your database goes offline and Apache cannot log to it, mod_log_mysql | ||
| 187 | intelligently preserves any queries to a local text file. (By | ||
| 188 | default the file is /tmp/mysql-preserve.) This will allow you to not | ||
| 189 | miss those entries; when you bring your database back online it is a | ||
| 190 | simple matter to import the contents of this preserve file. To do | ||
| 191 | this simply copy the file to your MySQL server and run an import | ||
| 192 | as follows: | ||
| 193 | # mysql -uadminuser -p mydbname < mysql-preserve | ||
| 194 | |||
| 195 | 108 | ||
| 196 | Author / Maintainer | 109 | Author / Maintainer |
| 197 | ------------------- | 110 | ------------------- |
| @@ -202,7 +115,7 @@ text modules, so all that credit goes to the Apache Server group. | |||
| 202 | The MySQL routines and directives were added by Zeev Suraski | 115 | The MySQL routines and directives were added by Zeev Suraski |
| 203 | <bourbon@netvision.net.il>. | 116 | <bourbon@netvision.net.il>. |
| 204 | 117 | ||
| 205 | Changes from 1.06 on and the new documentation were added by | 118 | All changes from 1.06+ and the new documentation were added by |
| 206 | Chris Powell <chris@grubbybaby.com>. It seems that the module had fallen | 119 | Chris Powell <chris@grubbybaby.com>. It seems that the module had fallen |
| 207 | into the "unmaintained" category -- it hadn't been updated since 1998 -- | 120 | into the "unmaintained" category -- it hadn't been updated since 1998 -- |
| 208 | so Chris adopted it as the new maintainer. | 121 | so Chris adopted it as the new maintainer. |
