|
If you were logged in you would be able to see more operations.
|
|
|
Load average and memory usage very high on wiley recently, due to webalizer.
djce@wiley:/usr/local/webstats/zaphod-nginx$ ls -lh tmp/carl/
total 4.1G
rw-rr- 1 root root 4.1G 2012-07-01 07:13 musicbrainz-full-combined.log-20120701
djce@wiley:/usr/local/webstats/zaphod-nginx$
Typical daily log size.
Possible approaches:
- suck it up
- use something other than webalizer
- find a way to make webalizer use less resources (feed the log in in chunks? adjust webalizer.conf ?)
- process less data (only feed in, say, 10% of the log at random)
- process the data elsewhere (not on wiley)
|
|
Description
|
Load average and memory usage very high on wiley recently, due to webalizer.
djce@wiley:/usr/local/webstats/zaphod-nginx$ ls -lh tmp/carl/
total 4.1G
rw-rr- 1 root root 4.1G 2012-07-01 07:13 musicbrainz-full-combined.log-20120701
djce@wiley:/usr/local/webstats/zaphod-nginx$
Typical daily log size.
Possible approaches:
- suck it up
- use something other than webalizer
- find a way to make webalizer use less resources (feed the log in in chunks? adjust webalizer.conf ?)
- process less data (only feed in, say, 10% of the log at random)
- process the data elsewhere (not on wiley)
|
Show » |
Sort Order:
|
Task
Open
Normal
webalizer is killing wiley
Might want to disable webalizer processing of "musicbrainz-full" for now, until we resolve this.
To do so, edit zaphod-nginx-webalizer/wiley/bin/update-all in git.musicbrainz.org:sysadmin/stats (comment out musicbrainz-full); commit, push, pull onto wiley. Be careful when re-enabling though because by default it will try to catch up on all the logs it missed.