diff options
Diffstat (limited to 'README')
-rw-r--r-- | README | 151 |
1 files changed, 32 insertions, 119 deletions
@@ -1,4 +1,4 @@ | |||
1 | $Id: README,v 1.4 2002/04/08 07:06:20 helios Exp $ | 1 | $Id: README,v 1.5 2002/04/21 23:01:53 helios Exp $ |
2 | 2 | ||
3 | 3 | ||
4 | Homepage | 4 | Homepage |
@@ -13,35 +13,25 @@ Approach | |||
13 | In order to save speed and overhead, links are kept alive in between | 13 | In order to save speed and overhead, links are kept alive in between |
14 | queries. This module uses one SQL link per httpd process. Among other | 14 | queries. This module uses one SQL link per httpd process. Among other |
15 | things, this means that this module supports logging into only one | 15 | things, this means that this module supports logging into only one |
16 | MySQL server, and for now, also, only one SQL database (although the | 16 | MySQL server, and for now, also, only one SQL database. |
17 | latter limitation can be relatively easily removed). | ||
18 | |||
19 | Different data can be sent to different tables. i.e., it's possible to | ||
20 | define one table for TransferLog, one for RefererLog, and a 3rd for | ||
21 | AgentLog. [ Note: this is now deprecated behavior. Please consider | ||
22 | logging Agent and Referer to the same table as your transfers. ] | ||
23 | 17 | ||
24 | Virtual hosts are supported in the same manner they are in the regular | 18 | Virtual hosts are supported in the same manner they are in the regular |
25 | logging modules. If you specify a different table for a virtual | 19 | logging modules. If you specify a different table for a virtual |
26 | host it will be used, otherwise the 'general' would be used. Note: | 20 | host it will be used, otherwise the 'general' would be used. |
27 | since all 3 types of logs are implemented within the same module, if | 21 | |
28 | you specify an overriding table for a virtual host for one type of log, | 22 | SQL links are opened by each child process when it is born. Error reporting |
29 | it'll ignore any previous 'general' defaults (see the example in the | 23 | is robust throughout and will let you know about database issues |
30 | end). | 24 | in the standard Apache error-log for the server or virtual server. |
31 | 25 | ||
32 | SQL links are opened on demand (i.e., the first time each httpd needs | 26 | A robust "preserve" capability has now been implemented as well. This |
33 | to log something to SQL, the link is opened). In case the SQL server | 27 | permits the module to preserve any failed INSERT commands to a local |
34 | is down when trying to connect to it, the module remains silent and | 28 | file on its machine. In any situation that the database is unavailable -- |
35 | logs no error (I didn't want thousands of error messages in the | 29 | e.g. the network fails, you reboot the machine, etc. -- mod_log_mysql |
36 | logfile). In case the SQL link is broken ("mysql server has gone | 30 | will note this in the error log and begin appending its log entries to |
37 | away") a proper error message is kept to the error log (textual :), and | 31 | the preserve file. At the time that your MySQL server returns to service, |
38 | the module tries to reestablish the concact (and reports whether it | 32 | each of these preserve files is easily imported because it is stored in SQL: |
39 | succeeded or not in the error log). If the link cannot be | 33 | |
40 | reestablished, the module will, again, remain silent. Technical note: | 34 | # mysql -uadminuser -p mydbname < /tmp/mysql-preserve |
41 | The SQL link is registered using apache's pool mechanism, so SQL links | ||
42 | are properly closed on any normal shutdown, kill -HUP or kill -TERM. | ||
43 | This also means that if you restart the MySQL daemon for any reason you | ||
44 | should restart Apache. | ||
45 | 35 | ||
46 | 36 | ||
47 | 37 | ||
@@ -63,85 +53,22 @@ What gets logged by default? | |||
63 | 53 | ||
64 | All the data that would be contained in the "Combined Log Format" | 54 | All the data that would be contained in the "Combined Log Format" |
65 | is logged by default, plus a little extra. Your best bet is to | 55 | is logged by default, plus a little extra. Your best bet is to |
66 | accept this default and employ the enclosed access_log.sql to | 56 | begin by accepting this default, then later customize the log |
67 | format your table. Customize your logging format after you've | 57 | configuration based on your needs. |
68 | had a chance to experiment with the default first. | 58 | |
69 | 59 | The online documentation of the run-time directives includes a full | |
70 | If you just want to log enough data to be able to reconstruct | 60 | explanation of what you can log, including examples. |
71 | a Combined Log Format log, log these: | ||
72 | |||
73 | +------------------+------------------+ | ||
74 | | Field | Type | | ||
75 | +------------------+------------------+ | ||
76 | | remote_host | varchar(50) | | ||
77 | | remote_user | varchar(50) | | ||
78 | | request_uri | varchar(50) | | ||
79 | | virtual_host | varchar(50) | | ||
80 | | time_stamp | int(10) unsigned | | ||
81 | | status | smallint(6) | | ||
82 | | bytes_sent | int(11) | | ||
83 | | referer | varchar(255) | | ||
84 | | agent | varchar(255) | | ||
85 | | request_method | varchar(6) | | ||
86 | | request_protocol | varchar(10) | | ||
87 | +------------------+------------------+ | ||
88 | |||
89 | remote_host: corresponds to the Apache %h directive. Contains the remote | ||
90 | hostname or IP of the machine accessing your server. | ||
91 | Example: si4002.inktomi.com | ||
92 | |||
93 | remote_user: corresponds to the Apache %u directive. Contains the | ||
94 | userid of people who have authenticated to your server, if applicable. | ||
95 | Example: freddy | ||
96 | |||
97 | request_uri: corresponds to the Apache %U directive. Contains the | ||
98 | URL path requested, excluding any query string. This is different than | ||
99 | the %r information you might be used to seeing: | ||
100 | |||
101 | %r: GET /cgi-bin/neomail.pl?sessionid=freddy-session-0.742143231719&sort=date_rev HTTP/1.1 | ||
102 | %U: /cgi-bin/neomail.pl | ||
103 | |||
104 | We log %U because it contains the real meat of the information that is | ||
105 | needed for log analysis, and saves the database a LOT of wasted growth | ||
106 | on unneeded bytes. | ||
107 | |||
108 | virtual_host: contains the VirtualHost that is making the log entry. This | ||
109 | allows you to log multiple VirtualHosts to a single MySQL database and | ||
110 | yet still be able to extract them for separate analysis. | ||
111 | Example: www.grubbybaby.com | ||
112 | |||
113 | time_stamp: contains the time that the request was logged. Please see | ||
114 | "Notes" below to get a better understanding of this. | ||
115 | Example: 1014249231 | ||
116 | |||
117 | status: corresponds to the Apache %t directive. Contains the HTTP status | ||
118 | of the request. | ||
119 | Example: 404 | ||
120 | |||
121 | bytes_sent: corresponds to the Apache %b directive. Contains the number | ||
122 | of bytes sent to service the request. | ||
123 | Example: 23123 | ||
124 | |||
125 | referer: corresponds to the Apache "%{Referer}i" directive. Contains the | ||
126 | referring HTML page's URL, if applicable. | ||
127 | Example: http://www.foobar.com/links.html | ||
128 | |||
129 | agent: corresponds to the Apache "%{User-Agent}" directive. Contains the | ||
130 | broswer type (user agent) of the software that made the request. | ||
131 | Example: Mozilla/3.0 (Slurp/si; slurp@inktomi.com; http://www.inktomi.com/slurp.html) | ||
132 | |||
133 | request_method: corresponds to the Apache %m directive. Contains the type | ||
134 | of request sent: GET, PUT, etc. | ||
135 | Example: GET | ||
136 | |||
137 | request_protocol: corresponds to the Apache %H directive. Contains the HTTP | ||
138 | protocol that was used. | ||
139 | Example: HTTP/1.1 | ||
140 | 61 | ||
141 | 62 | ||
142 | Notes | 63 | Notes |
143 | ----- | 64 | ----- |
144 | 65 | ||
66 | * You will customarily set most of your run-time configuration directives | ||
67 | on a per-virtualserver basis, with only MySQLMassVirtualHosting, | ||
68 | MySQLLoginInfo and MySQLDatabase 'outside' in the main server config. | ||
69 | Any directives other than those in the main config do NOT get inherited | ||
70 | by the virutal servers. | ||
71 | |||
145 | * The 'time_stamp' field is stored in an UNSIGNED INTEGER column, in the | 72 | * The 'time_stamp' field is stored in an UNSIGNED INTEGER column, in the |
146 | standard unix "seconds since 1/1/1970 12:00:00" format. This is | 73 | standard unix "seconds since 1/1/1970 12:00:00" format. This is |
147 | superior to storing the access time as a string due to size | 74 | superior to storing the access time as a string due to size |
@@ -161,20 +88,15 @@ Notes | |||
161 | Log Format compliant. You can then feed this to your favorite web | 88 | Log Format compliant. You can then feed this to your favorite web |
162 | log analysis tool. | 89 | log analysis tool. |
163 | 90 | ||
164 | |||
165 | * The table's string values can be CHAR or VARCHAR, at a length of your choice. | 91 | * The table's string values can be CHAR or VARCHAR, at a length of your choice. |
166 | VARCHAR is superior because it truncates long strings; CHAR types are | 92 | VARCHAR is superior because it truncates long strings; CHAR types are |
167 | fixed-length and will be padded with spaces. Just like the | 93 | fixed-length and will be padded with spaces. Just like the |
168 | time_stamp described above, that kind of space waste will add up over | 94 | time_stamp described above, that kind of space waste will add up over |
169 | thousands of records. | 95 | thousands of records. |
170 | 96 | ||
171 | 97 | * Be careful not to go overboard setting fields to NOT NULL. If a field is | |
172 | * Most fields should probably be set to NOT NULL. The only ones that | 98 | marked NOT NULL then it must contain data in the INSERT or the INSERT |
173 | shouldn't are extra fields that you don't intend the logging module | 99 | will fail. |
174 | to update. (You can have other fields in the logging tables if you'd | ||
175 | like, but if they're set to NOT NULL then the logging module won't be | ||
176 | able to insert rows to these tables.) | ||
177 | |||
178 | 100 | ||
179 | * Apache normally logs numeric fields with a '-' character to mean "not | 101 | * Apache normally logs numeric fields with a '-' character to mean "not |
180 | applicable," e.g. bytes_sent on a request with a 304 response code. | 102 | applicable," e.g. bytes_sent on a request with a 304 response code. |
@@ -183,15 +105,6 @@ Notes | |||
183 | makes perfect sense anyway. | 105 | makes perfect sense anyway. |
184 | 106 | ||
185 | 107 | ||
186 | * If your database goes offline and Apache cannot log to it, mod_log_mysql | ||
187 | intelligently preserves any queries to a local text file. (By | ||
188 | default the file is /tmp/mysql-preserve.) This will allow you to not | ||
189 | miss those entries; when you bring your database back online it is a | ||
190 | simple matter to import the contents of this preserve file. To do | ||
191 | this simply copy the file to your MySQL server and run an import | ||
192 | as follows: | ||
193 | # mysql -uadminuser -p mydbname < mysql-preserve | ||
194 | |||
195 | 108 | ||
196 | Author / Maintainer | 109 | Author / Maintainer |
197 | ------------------- | 110 | ------------------- |
@@ -202,7 +115,7 @@ text modules, so all that credit goes to the Apache Server group. | |||
202 | The MySQL routines and directives were added by Zeev Suraski | 115 | The MySQL routines and directives were added by Zeev Suraski |
203 | <bourbon@netvision.net.il>. | 116 | <bourbon@netvision.net.il>. |
204 | 117 | ||
205 | Changes from 1.06 on and the new documentation were added by | 118 | All changes from 1.06+ and the new documentation were added by |
206 | Chris Powell <chris@grubbybaby.com>. It seems that the module had fallen | 119 | Chris Powell <chris@grubbybaby.com>. It seems that the module had fallen |
207 | into the "unmaintained" category -- it hadn't been updated since 1998 -- | 120 | into the "unmaintained" category -- it hadn't been updated since 1998 -- |
208 | so Chris adopted it as the new maintainer. | 121 | so Chris adopted it as the new maintainer. |