summaryrefslogtreecommitdiffstats
path: root/README
diff options
context:
space:
mode:
authorGravatar Christopher Powell 2002-04-21 23:01:53 +0000
committerGravatar Christopher Powell 2002-04-21 23:01:53 +0000
commitb63c5d2438aabf0d7721c38387995cb4fb98345f (patch)
tree73f99f046bb7daae711c5a7bdfe0f89677e22bda /README
parent13e1ecc20c883f71b54bf1fe09488b028e51171e (diff)
Significant bugfixes and feature additions on the way to 1.16...
Diffstat (limited to 'README')
-rw-r--r--README151
1 files changed, 32 insertions, 119 deletions
diff --git a/README b/README
index 977080f..7bea822 100644
--- a/README
+++ b/README
@@ -1,4 +1,4 @@
1$Id: README,v 1.4 2002/04/08 07:06:20 helios Exp $ 1$Id: README,v 1.5 2002/04/21 23:01:53 helios Exp $
2 2
3 3
4Homepage 4Homepage
@@ -13,35 +13,25 @@ Approach
13In order to save speed and overhead, links are kept alive in between 13In order to save speed and overhead, links are kept alive in between
14queries. This module uses one SQL link per httpd process. Among other 14queries. This module uses one SQL link per httpd process. Among other
15things, this means that this module supports logging into only one 15things, this means that this module supports logging into only one
16MySQL server, and for now, also, only one SQL database (although the 16MySQL server, and for now, also, only one SQL database.
17latter limitation can be relatively easily removed).
18
19Different data can be sent to different tables. i.e., it's possible to
20define one table for TransferLog, one for RefererLog, and a 3rd for
21AgentLog. [ Note: this is now deprecated behavior. Please consider
22logging Agent and Referer to the same table as your transfers. ]
23 17
24Virtual hosts are supported in the same manner they are in the regular 18Virtual hosts are supported in the same manner they are in the regular
25logging modules. If you specify a different table for a virtual 19logging modules. If you specify a different table for a virtual
26host it will be used, otherwise the 'general' would be used. Note: 20host it will be used, otherwise the 'general' would be used.
27since all 3 types of logs are implemented within the same module, if 21
28you specify an overriding table for a virtual host for one type of log, 22SQL links are opened by each child process when it is born. Error reporting
29it'll ignore any previous 'general' defaults (see the example in the 23is robust throughout and will let you know about database issues
30end). 24in the standard Apache error-log for the server or virtual server.
31 25
32SQL links are opened on demand (i.e., the first time each httpd needs 26A robust "preserve" capability has now been implemented as well. This
33to log something to SQL, the link is opened). In case the SQL server 27permits the module to preserve any failed INSERT commands to a local
34is down when trying to connect to it, the module remains silent and 28file on its machine. In any situation that the database is unavailable --
35logs no error (I didn't want thousands of error messages in the 29e.g. the network fails, you reboot the machine, etc. -- mod_log_mysql
36logfile). In case the SQL link is broken ("mysql server has gone 30will note this in the error log and begin appending its log entries to
37away") a proper error message is kept to the error log (textual :), and 31the preserve file. At the time that your MySQL server returns to service,
38the module tries to reestablish the concact (and reports whether it 32each of these preserve files is easily imported because it is stored in SQL:
39succeeded or not in the error log). If the link cannot be 33
40reestablished, the module will, again, remain silent. Technical note: 34 # mysql -uadminuser -p mydbname < /tmp/mysql-preserve
41The SQL link is registered using apache's pool mechanism, so SQL links
42are properly closed on any normal shutdown, kill -HUP or kill -TERM.
43This also means that if you restart the MySQL daemon for any reason you
44should restart Apache.
45 35
46 36
47 37
@@ -63,85 +53,22 @@ What gets logged by default?
63 53
64All the data that would be contained in the "Combined Log Format" 54All the data that would be contained in the "Combined Log Format"
65is logged by default, plus a little extra. Your best bet is to 55is logged by default, plus a little extra. Your best bet is to
66accept this default and employ the enclosed access_log.sql to 56begin by accepting this default, then later customize the log
67format your table. Customize your logging format after you've 57configuration based on your needs.
68had a chance to experiment with the default first. 58
69 59The online documentation of the run-time directives includes a full
70If you just want to log enough data to be able to reconstruct 60explanation of what you can log, including examples.
71a Combined Log Format log, log these:
72
73+------------------+------------------+
74| Field | Type |
75+------------------+------------------+
76| remote_host | varchar(50) |
77| remote_user | varchar(50) |
78| request_uri | varchar(50) |
79| virtual_host | varchar(50) |
80| time_stamp | int(10) unsigned |
81| status | smallint(6) |
82| bytes_sent | int(11) |
83| referer | varchar(255) |
84| agent | varchar(255) |
85| request_method | varchar(6) |
86| request_protocol | varchar(10) |
87+------------------+------------------+
88
89remote_host: corresponds to the Apache %h directive. Contains the remote
90 hostname or IP of the machine accessing your server.
91 Example: si4002.inktomi.com
92
93remote_user: corresponds to the Apache %u directive. Contains the
94 userid of people who have authenticated to your server, if applicable.
95 Example: freddy
96
97request_uri: corresponds to the Apache %U directive. Contains the
98 URL path requested, excluding any query string. This is different than
99 the %r information you might be used to seeing:
100
101 %r: GET /cgi-bin/neomail.pl?sessionid=freddy-session-0.742143231719&sort=date_rev HTTP/1.1
102 %U: /cgi-bin/neomail.pl
103
104 We log %U because it contains the real meat of the information that is
105 needed for log analysis, and saves the database a LOT of wasted growth
106 on unneeded bytes.
107
108virtual_host: contains the VirtualHost that is making the log entry. This
109 allows you to log multiple VirtualHosts to a single MySQL database and
110 yet still be able to extract them for separate analysis.
111 Example: www.grubbybaby.com
112
113time_stamp: contains the time that the request was logged. Please see
114 "Notes" below to get a better understanding of this.
115 Example: 1014249231
116
117status: corresponds to the Apache %t directive. Contains the HTTP status
118 of the request.
119 Example: 404
120
121bytes_sent: corresponds to the Apache %b directive. Contains the number
122 of bytes sent to service the request.
123 Example: 23123
124
125referer: corresponds to the Apache "%{Referer}i" directive. Contains the
126 referring HTML page's URL, if applicable.
127 Example: http://www.foobar.com/links.html
128
129agent: corresponds to the Apache "%{User-Agent}" directive. Contains the
130 broswer type (user agent) of the software that made the request.
131 Example: Mozilla/3.0 (Slurp/si; slurp@inktomi.com; http://www.inktomi.com/slurp.html)
132
133request_method: corresponds to the Apache %m directive. Contains the type
134 of request sent: GET, PUT, etc.
135 Example: GET
136
137request_protocol: corresponds to the Apache %H directive. Contains the HTTP
138 protocol that was used.
139 Example: HTTP/1.1
140 61
141 62
142Notes 63Notes
143----- 64-----
144 65
66* You will customarily set most of your run-time configuration directives
67 on a per-virtualserver basis, with only MySQLMassVirtualHosting,
68 MySQLLoginInfo and MySQLDatabase 'outside' in the main server config.
69 Any directives other than those in the main config do NOT get inherited
70 by the virutal servers.
71
145* The 'time_stamp' field is stored in an UNSIGNED INTEGER column, in the 72* The 'time_stamp' field is stored in an UNSIGNED INTEGER column, in the
146 standard unix "seconds since 1/1/1970 12:00:00" format. This is 73 standard unix "seconds since 1/1/1970 12:00:00" format. This is
147 superior to storing the access time as a string due to size 74 superior to storing the access time as a string due to size
@@ -161,20 +88,15 @@ Notes
161 Log Format compliant. You can then feed this to your favorite web 88 Log Format compliant. You can then feed this to your favorite web
162 log analysis tool. 89 log analysis tool.
163 90
164
165* The table's string values can be CHAR or VARCHAR, at a length of your choice. 91* The table's string values can be CHAR or VARCHAR, at a length of your choice.
166 VARCHAR is superior because it truncates long strings; CHAR types are 92 VARCHAR is superior because it truncates long strings; CHAR types are
167 fixed-length and will be padded with spaces. Just like the 93 fixed-length and will be padded with spaces. Just like the
168 time_stamp described above, that kind of space waste will add up over 94 time_stamp described above, that kind of space waste will add up over
169 thousands of records. 95 thousands of records.
170 96
171 97* Be careful not to go overboard setting fields to NOT NULL. If a field is
172* Most fields should probably be set to NOT NULL. The only ones that 98 marked NOT NULL then it must contain data in the INSERT or the INSERT
173 shouldn't are extra fields that you don't intend the logging module 99 will fail.
174 to update. (You can have other fields in the logging tables if you'd
175 like, but if they're set to NOT NULL then the logging module won't be
176 able to insert rows to these tables.)
177
178 100
179* Apache normally logs numeric fields with a '-' character to mean "not 101* Apache normally logs numeric fields with a '-' character to mean "not
180 applicable," e.g. bytes_sent on a request with a 304 response code. 102 applicable," e.g. bytes_sent on a request with a 304 response code.
@@ -183,15 +105,6 @@ Notes
183 makes perfect sense anyway. 105 makes perfect sense anyway.
184 106
185 107
186* If your database goes offline and Apache cannot log to it, mod_log_mysql
187 intelligently preserves any queries to a local text file. (By
188 default the file is /tmp/mysql-preserve.) This will allow you to not
189 miss those entries; when you bring your database back online it is a
190 simple matter to import the contents of this preserve file. To do
191 this simply copy the file to your MySQL server and run an import
192 as follows:
193 # mysql -uadminuser -p mydbname < mysql-preserve
194
195 108
196Author / Maintainer 109Author / Maintainer
197------------------- 110-------------------
@@ -202,7 +115,7 @@ text modules, so all that credit goes to the Apache Server group.
202The MySQL routines and directives were added by Zeev Suraski 115The MySQL routines and directives were added by Zeev Suraski
203<bourbon@netvision.net.il>. 116<bourbon@netvision.net.il>.
204 117
205Changes from 1.06 on and the new documentation were added by 118All changes from 1.06+ and the new documentation were added by
206Chris Powell <chris@grubbybaby.com>. It seems that the module had fallen 119Chris Powell <chris@grubbybaby.com>. It seems that the module had fallen
207into the "unmaintained" category -- it hadn't been updated since 1998 -- 120into the "unmaintained" category -- it hadn't been updated since 1998 --
208so Chris adopted it as the new maintainer. 121so Chris adopted it as the new maintainer.