Funky redo behaviour in 10g

I'll admit it, I don't have that much exposure to 10g just yet. I'm excited about it, I just don't have much experience on it.

We put up a development database on 10.2 a couple months back just so we could start getting experience on 10g. As we add more and more development schemas to this database, we're getting a taste for the new features.

We've been getting a disproportionate number of "Checkpoint not complete" messages lately. Sure, there's pretty heavy write activity going on at the time, but not enough to fill all my redo logs. I have six groups of 128M logfiles with two members each. During the heavy write activity, my logs get switched a few times in a minute. However, I know my log_archive_dest can keep up with the write activity, so there should be no problem archiving them that fast. From the alert.log:

Thu Dec 29 19:05:37 2005
Thread 1 cannot allocate new log, sequence 9945
Private strand flush not complete
Current log# 4 seq# 9944 mem# 0: /u01/oradata/demo1/redo04_01.log
Current log# 4 seq# 9944 mem# 1: /u02/oradata/demo1/redo04_02.log
Thread 1 advanced to log sequence 9945
Current log# 5 seq# 9945 mem# 0: /u01/oradata/demo1/redo05_01.log
Current log# 5 seq# 9945 mem# 1: /u02/oradata/demo1/redo05_02.log
Thu Dec 29 19:06:28 2005
Thread 1 advanced to log sequence 9946
Current log# 1 seq# 9946 mem# 0: /u01/oradata/demo1/redo01_01.log
Current log# 1 seq# 9946 mem# 1: /u02/oradata/demo1/redo01_02.log
Thu Dec 29 19:06:42 2005
Thread 1 advanced to log sequence 9947
Current log# 2 seq# 9947 mem# 0: /u01/oradata/demo1/redo02_01.log
Current log# 2 seq# 9947 mem# 1: /u02/oradata/demo1/redo02_02.log
Thu Dec 29 19:07:00 2005
Thread 1 cannot allocate new log, sequence 9948
Checkpoint not complete
Current log# 2 seq# 9947 mem# 0: /u01/oradata/demo1/redo02_01.log
Current log# 2 seq# 9947 mem# 1: /u02/oradata/demo1/redo02_02.log
Thread 1 advanced to log sequence 9948

After the write activity is over, I look at v$logfile and v$log and see that several groups are still ACTIVE.

I then doubled the size of the logfiles and reran the big job that caused the "Checkpoint not complete" error. While the job didn't cause a "Checkpoint not complete" error, it did make three of the log groups ACTIVE, which I kind of expected. However, 20-30 minutes later, I looked at v$logfile and v$log to see almost all my log groups were either ACTIVE or CURRENT.

system@demo1.us> l
1  select l.group#, lf.member, l.bytes/1024/1024 mb,  l.status, l.archived
2  from v$logfile lf, v$log l
3* where l.group# = lf.group#

GROUP# MEMBER                             MB STATUS     ARC
------ ------------------------------- ----- ---------- ---
   1 /u01/oradata/demo1/redo01_01.log   256 CURRENT    NO
   3 /u01/oradata/demo1/redo03_01.log   256 ACTIVE     YES
   3 /u02/oradata/demo1/redo03_02.log   256 ACTIVE     YES
   4 /u01/oradata/demo1/redo04_01.log   256 ACTIVE     YES
   4 /u02/oradata/demo1/redo04_02.log   256 ACTIVE     YES
   5 /u01/oradata/demo1/redo05_01.log   256 ACTIVE     YES
   5 /u02/oradata/demo1/redo05_02.log   256 ACTIVE     YES
   2 /u01/oradata/demo1/redo02_01.log   256 INACTIVE   YES
   2 /u02/oradata/demo1/redo02_02.log   256 INACTIVE   YES
   1 /u02/oradata/demo1/redo01_02.log   256 CURRENT    NO
   6 /u01/oradata/demo1/redo06_01.log   256 ACTIVE     YES
   6 /u02/oradata/demo1/redo06_02.log   256 ACTIVE     YES
                                      -----
Total                                   3072

12 rows selected.

Elapsed: 00:00:00.05

During light load, I would expect one group CURRENT and maybe one group ACTIVE. The other funny thing is the logs show they are archived, which if I look at the log_archive_dest, they are. I have to do some more research, but I suspect there's an init.ora parameter or something that isn't set correctly...

Router and WAP in one

Santa brought me a new NetGear router for Christmas. This particular router has an 802.11g WAP built-in so I don't have to give up an ethernet port for it. The basic setup was really easy and when I customized things like the LAN IP and DHCP addresses it was somewhat easy. The only thing that was somewhat difficult was setting up the WEP. I tried to choose my former Passphrase so I wouldn't have to change the wireless devices, but the router just wouldn't take it because it was the wrong length. The WAP has the ability to run WPA, so I set that up instead. Unfortunately, using WPA precludes me from using my Palm, but browsing on the Palm never really worked well anyway.

The old Linksys WAP is on it's way to the local landfill where it belongs. I'm hanging on to the old Linksys router for a backup.

Interesting take on RTFM

I got a kick out of Andrew Gilfster's blog about RTFM. Maybe I'm a little more cynical, cause I think it's perfectly appropriate in certain aggrevated circumstances to give somebody "RTFM" as an answer.

The "New & Improved" Metalink

I jumped on the new metalink last night and was zipping along pretty good. It's definitely different, but I'm not sure different is bad. Today, however, 6 minutes to login. Three minutes to bring up the TAR query screen (or should I say Service Request). Another 4 minutes to get my TAR back. If they're going to make Metalink the primary interface to customer service, can we at least make sure it is somewhat responsive?

Yummy yum

One of the things that pushed me to upgrade to Fedora Core 4 was suspense of the automatic updates from the Red Hat Network a while ago. It seemed as though every time I wanted to install something I needed to search out the correct RPMs and install them before I could intstall the software I originally wanted to install.

FC4 opened my eyes to a totally new updating method; yum. I did a quick search and found The Unoffical Fedora FAQ which walked me through setting up yum and installing software for it. I installed the updated gcc compiler using yum and it only took a couple minutes to download gcc and any dependencies. "Wow, this is easy", I thought to myself, so I used yum to update OpenOffice.org to 2.0. This is definitely a nice tool to keep some of your general software up to date.

I don't think I'll use yum to keep my main database software updated. For example, I'm going to be doing research on having multiple versions of MySQL on the same system. During this research, I'm going to put MySQL 4.1 in one directory and 5.0 in another. I don't want my 4.1 software automatically updated until I want it updated.

I wonder if Oracle has a yum repository for Oracle Applications? (If you deal with Oracle Apps, you know what I'm talking about).

I think this will be a big time-saver.

FC4 installed

I have the base Fedora Core 4 installed with only a couple issues. When I decided to install the new OS and update my software, I figured I might as well upgrade my Hard Drive. I got an Ultra ATA/133 200G drive and replaced my 20G drive. Of course, I didn't read the package and didn't notice I needed a different I/O card. Back to CompUSA to get the new card.

The second little glitch was I downloaded the ISO images and burned them to CD. Except I didn't realized I had to burn them in a specific way. The second time I figured it out and was able to install right off CD.

The particular distribution I used had Firefox 1.0.4 and OpenOffice 1.9 (2.0 beta) so I'm no further ahead in that respect. Since I was planning on re-installing these packages anyway, it's not a big deal. My first task, however, is to get the MySQL 5.0 source and get Apache configured. More updates to come...

Out with the old, in with the new

I'll have a couple free hours over the next few weeks, so I have decided to upgrade my Linux box at home to RedHat Fedora Core 4. The software on it was really out of date (Firefox 1.0.4, Thunderbird 0.9, MySQL 4.0, OpenOffice 1.0, etc.), so instead of updating everything I figured I'd go the extra step and do everything. Tonight I am spending a couple hours downloading ISO images and burning them to CD. Stay tuned for the progress...

MySQL and icc solved

Thanks to Lenz Grimmer on the MySQL forums. His solution was to use the following variables:

CC=icc \
CFLAGS="-O3 -unroll2 -ip -mp -restrict" \
CXX=icpc \
CXXFLAGS="-O3 -unroll2 -ip -mp -restrict" \
LDFLAGS="-static-libcxa -i-static"

Installing just the MySQL Client using icc

I started out today doing what I thought was a simple task; Install the MySQL client using Intel's compiler. In my environment, we like to install the client software so it is generally available to everyone and the server software on the actual physical box. We feel this gives us the flexibility to upgrade the servers indendently from all the programs.

I've compiled the 5.0.16 software using gcc about a dozen times already and other than a few simple tweaks here and there, it was a no-brainer. When I installed the client, I used gcc instead of the preferred (in my environment) icc.

I started by issuing a "make clean" to remove any builds I did before. I then changed my CC and CXX environment variables to point to the icc compiler and did:

$ configure --without-server --prefix=/usr/local/mysql_client

On my first try I encountered errors that were looking for "__cxa_pure_virtual". I googled and found this which gave me some additional things to do. Sure enough, this solved my "__cxa_pure_virtual" problem, but now I have more errors:

$ export CC=icc
$ export CXX=icc
$ export CFLAGS="-O3 -unroll2 -ip -mp -no-gcc -restrict"
$ export CXXFLAGS="-O3 -unroll2 -ip -mp -no-gcc -restrict"

and then I configure with the following options:

$ ./configure --without-server --prefix=/usr/local/mysql_client --enable-thread-safe-client --enable-local-infile --enable-assembler --disable-shared --with-client-ldflags=-all-static

I then run "make clean" to clean out any failed compiles and it runs fine.

I then run just "make" and it chugs along for a while, but then exits with errors like:

mkdir .libs
icc -O3 -DDBUG_OFF -O3 -unroll2 -ip -mp -no-gcc -restrict
 -o gen_lex_hash gen_lex_hash.o ../myisam/libmyisam.a
 ../myisammrg/libmyisammrg.a
../heap/libheap.a ../vio/libvio.a ../mysys/libmysys.a ../dbug/libdbug.a
../regex/libregex.a ../strings/libmystrings.a -lz
 -lpthread -lcrypt -lnsl -lm -lpthread
gen_lex_hash.o(.eh_frame+0x12): undefined reference to
 `__gxx_personality_v0'
make[2]: *** [gen_lex_hash] Error 1
make[2]: Leaving directory `/srcpath/5.0.16/mysql-5.0.16/sql'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/srcpath/5.0.16/mysql-5.0.16'
make: *** [all] Error 2

Obviously, I'm missing a reference to __gxx_personality_v0, but where do I get that from? Maybe one of my other libaries is out of date? If anybody has any hints, I'd appreciate you sharing them at http://forums.mysql.com/read.php?11,59295,59295#msg-59295.

Update: Solution

Where MySQL and Oracle are the same...

Say you've developed software on Oracle for a while. You know that if you start a query in SQL*Plus and then kill -9 the process that your query is still running on the server. You've got enough calls from your DBA to know that if you do that, you better call and let him know it's still running. Would you think it's any different on MySQL? Nope.

Three wishes...

While looking at something else, I came across 3 Things I wish I'd done by Jeanne Sahadi. While Jeanne writes about money and finance related stuff, I think the third point about valuing your work is right on.

Early in my career, I worked for a very large university in their IT department. Just being out of college, I was very grateful to even have a job. But anybody who works in the public sector knows, the pay ain't that great. In order to make ends meet, I started writing code "on the side" with a friend of a friend. I wrote a system in Paradox (version 3.5 for you techno geeks out there) for $10/hr. I got paid, but found out later that the person I was working for billed the client $50/hr for my time. This person then went on to sell this system to other clients and, of course, I didn't have any rights to it.

I also wish I listened to my gut. I had been working at a regional telecom company in the late 1990's. I had been perfectly happy there for three years and the pay was more than adequate. In 1998, during the rise of the dotcom bubble, I was seduced by a consulting company by the promise of 100% bonuses and four day work weeks. I agonized about it for about a week, and although my gut told me it was too good to be true, I jumped ship. After about two weeks I knew I made a mistake, but told myself it would get better. Fortunately, I had a good management team back at the telecom company and kept in touch with them. After two months I decided I'd had enough of consulting and went back to my old job.

Moving an application

We moved one of our major schemas to a new database this past weekend. This schema had lived in the same database for over six years. Over that time many people worked on the system and brought their experience to the development of it. Some of the things we found during the move:

1. Some Java programs still continue to use a connection URL of host:port:sid.
2. People really like synonyms. There were a couple situations where a synonym pointed to a synonym which pointed to a synonym which pointed to a synonym...
3. Recently, we have been setting up TNS aliases for each application and not each database. Sure, when we have multiple applications on one database there are multiple aliases. But when we move the application and flip the alias, everybody changes at once. Programs that used the application specific aliases moved with no problems.
4. It's 2005 (almost 2006) and we are still fighting people hard-coding username/passwords in their programs.
5. Got a call from a user Sunday, "I can't login with user X to the database." I replied, "User X doesn't live on that database anymore, and where did you get user X's password?"

It may be early...

It is really early in the College Basketball season, but who would have thought the North Carolina Tarheels would give the Illini a strong fight and be 4-1 after seeing Illinois AND Kentucky? Just when everyone counted old Roy out...

Support Survey

https://feedback.oracle.com/pls/feedback/feedback_ossdata.q22006_form