Moving logs (not archive logs or redo logs)

Question

My employer has six Exadatas: three in a RAC in one data center, and the other three in a RAC in another data center. One is primary, and the other is a DataGuard standby. On the standby RAC, we also have a couple stand-alone instances (one for training and another for testing). Yes, this is not at all ideal. It was done well before I started working at this place, and we’re working on undoing these sins of the past. To make it more fun, we’re planning on adding a few more stand-alone instances on the standby Exadatas.

This brings me to my question: the / and /u01 volumes are embarrassingly small (for example, each / is 30GB). We get occasional alerts about space approaching warning limits so I have to clear old log files out. I’ve got a job for this that’s been running for a year, but lately we’re getting alerts more frequently. I know this will get worse once we add more instances. I don’t think we have much more than 2 or 3 GB on each Exadata to expand the LVM volume for / so I was thinking of moving some of the log files (like Exawatcher) to an NFS export on our Oracle ZFS appliance. When I say “move some of the log files” I mean where they’re written to now would actually become a mount point on the ZFS box so the log files are completely off the Exadata’s file systems. Ultimately, I’d do the same for listener and ASM trace files as well. If this is not advisable/best practice, recommendations are appreciated!

Also of note is while I have experience as a Linux sysadmin, I and my coworkers together have probably the equivalent of 6 months’ worth of full-time experience with Oracle database administration.

Answer 1

You want to make sure that any network-mounted solution is also highly available. Loss of access to some log files could result in Oracle services locking up.

In the past I have also used logrotate to roll, compress, and archive (to network storage) or delete log files automatically from the file system. I'd probably try this before using NFS mount points to "move" the logs entirely. Rotate or archive as frequently as necessary and move the archives to the network storage. I also used Splunk to read most of those files in real time so that I didn't lose easy/quick access to any of the data as things were rotated.