Virus'

I've been battling the "Vundo" virus on a machine I was asked to work on for a family member. Somewhere between the 13th and 14th time scanning with Trend Micro's Internet Security, my mind began to wander.

Exactly what type of person writes a virus intent on inflicting aggravation on countless people? Are virus programmers the same people that cut you off in traffic? Are they the type of people that beat puppies?

I bet there are plenty of virus writers who think it's funny when a bus blows up.

I finally came to the conclusion that these people are lower than dog shit. I hope they can carry on a conversation with Saddam Hussein when they meet him.

Pick Two

There's a saying in the computer business, "Good, Fast, Cheap. Pick Two." This saying represents the trade offs we have to make every day on the front lines of IT.

Recently, we have started playing around with MySQL Enterprise Monitor (MEM). MEM is MySQL's offering for an event driven monitoring application similar to Oracle's Enterprise Manager. There are four pieces to MEM; the monitoring agent that gathers that db statistics, the query analyzer that allows you to "trace" queries, the connection proxy that allows you to redirect connections, and the data repository.

Our first stab at configuring MEM was putting the monitoring agent, query analyzer, and query proxy on the host being monitored with the repository db on a different machine. This actually worked pretty well until we had some big queries run through the query proxy. CPU utilization spiked because both the query proxy and the database had to handle the same set of data.

So MySQL Support suggested we move the proxy, query analyzer, and monitoring agent to a separate host. This "middle-tier" would handle the extra CPU load we were generating and keep our db host performance consistent. This solution worked very well, except we weren't getting host related statistics like CPU utilization on the DB host.

No problem, right? We moved the monitoring agent to the db host and kept the proxy and query analyzer on the middle-tier. Now we had our host related statistics, except we couldn't get the query analyzer to work because it has to have a monitoring agent present in order to work properly.

The only solution MySQL Support can offer us at this point is to have two agents monitoring the same database; one on the middle-tier host and one on the db host. OK, we'll do what needs to be done, but I have two fundamental problems with that configuration. First, my database background tells me that duplication will come back and bite you in the ass someday. My other very practical issue is now we are storing twice as much data in the repository for 10% more information.

So while the concept of MEM is great, the actual implementation is a compromise.

Note: Some of the above configurations are not currently supported by MySQL. Use at your own risk.

SGA Latch Contention

We had been getting a pretty persistent "WARNING: inbound connection timed out (ORA-3136)" error thrown to the alert.log on one of our 10.2.0.4 dbs. This error wasn't received once or twice, but about 40 times within a 2 minute window.

I did the research on it and found some sources that suggested my SQLNET.INBOUND_CONNECT_TIMEOUT needed to be adjusted upwards from the default value of 60 to 120 or something higher. I was pretty sure that wasn't the case as it was already set at 120. For the sake of progress, I bumped the timeout to 300 (5 minutes) and monitored closely.

About two days later, we received the same warning message in the alert.log file, except this time it happened about 90 times in a two minute period. Aha, a change. So that told me that something was going wrong during the authentication phase since increasing the connection timeout made the problem worse.

During the last occurrence, I also noticed that the mman process was consuming 100% of the CPU. At the time I wasn't sure if it was a symptom or a byproduct of the problem. However, as soon as the problem went away, the ora_mman process went back down to nearly 0%.

I setup a script to automatically do three systemstate dumps a minute apart when the mman process went to 100%. Oracle Support was able to tell me that during this time period, a bunch of my SQL had been invalidated as was waiting on a latch to be loaded back into the shared_pool. They also indicated that this massive amount of reparsing could happen because somebody did a DDL on a popular object or somebody flushed the shared pool.

I was pretty sure nobody flushed the shared pool, but that got me to thinking what would happen if Oracle shrank my shared pool due to the automatic memory management? I checked v$sga_resize_ops and found that around the times of my warning message, the shared_pool was being re-sized down in size. I brought this up to the support analyst and he suggested I set the shared_pool to the maximum size Oracle had re-sized it to.

That was three days ago, and we haven't had a warning since.

The theory is that while the shared_pool was being re-sized in the SGA, Oracle grabbed a latch. The memory resize operation took a while and while Oracle held that latch, nobody could login. Interesting theory, we'll see if it holds.

PC Support for Oracle

So I'm working with Oracle Support on an Oracle Applications R12 issue. They want to see some data from one of our tables and give me a query to run. I run the query in SQL Developer and export the results as a .csv file and upload it to the TAR.

Three days later they get back to me and say they can't open the files. Hmm, that's funny, let me try the same thing.

I load it into Open Office in about 2 seconds, no problem.

Then I load it into a database using SQL*Loader with no issues.

Same thing with MySQL and LOAD DATA, no issues.

I mail the file to myself and again it works no problem.

I send the file to a collegue and Open Office works for them as well.

The support analyst asks for a plain .xls worksheet so I try to load it into MS Excel. Bingo, I get an error:

This error is usually encountered when an attempt to open a file with more than 65,536 rows or 256 columns is made. Excel is limited to 65,536 rows of data and 256 columns per worksheet. You can have many worksheets with this number of rows and columns, but they are usually capable of fitting into one workbook (file). The number of worksheets you can have per workbook is limited only by the amount of available memory your system has. By default, Excel can manage 3 worksheets, more if there is available memory to support the quantity of data.

Truncation of rows or columns in excess of the limit is automatic and is not configurable. This issue can usually be remedied by opening the source file with a text editor, such as Microsoft Office Word, and then saving the file off into multiple files with row or column counts within the limits of an Excel worksheet. These files can then be opened or imported into Excel worksheets.

If you are using a data format that does not support use of a text editor, it may be easier to import the data into Microsoft Office Access and then use the export feature of Access to import the data to an Excel format. Other methods of importing large source material into multiple worksheets are available, but may be more complex than using either a text editor or Access.

Aha! MS Excel can't handle anything with over 256 columns of data and my data is 325 columns! (Not my design, it comes from Oracle Applications).

I post my findings to the TAR and suggest they load the data into a piece of software that can handle 325 columns, like maybe Oracle. (I didn't have the heart to suggest MySQL, but I guess they wouldn't take that as a slight anymore ...)

Oracle vs. SQL Server

information_schema.tables.table_rows

Here's something to be aware of. The table_rows column in the information_schema.tables table does not accurately reflect the number of rows in the actual table.

This weekend I was moving a db from one host to another. As a good DBA, I took the row counts from each table I was moving so I could check it on the other side. When the copy was done, I queried information_schema.tables again and dumped the results to a file. Running the two files through diff, I found that almost every table had a different number of rows.

I knew that couldn't be possible because I used a method where nobody had access to these tables when I was doing the copy.

On serverA, the information_schema reported that table1 had 2337 rows. On serverB, the information_schema reported that table1 had 2341 rows after the move. Doing a "SELECT count(*) FROM table1" yeilded 2343 rows on both serverA and serverB.

Some Helpful Drupal Resources

The best resource for understanding how Drupal works is actually the handbooks at Drupal.org.

Learn By The Drop has some very interesting videos about setting up Drupal and some other basic info to get your site up and working quickly. I browsed their screencast about setting up FCKEditor and was very impressed.

There are also some very interesting screencasts at DrupalTherapy.com. In fact, I used the Date + Calendar screencast to setup my own event content type.

Lastly, a blog aggregator called PlanetDrupal is also fun to keep up with.

What are some of your favorite Drupal resources?