Issue with Derby when using with IBM WebSphere Application Server Community Edition

[Suraj Batuwana]

Hi,

I have received a server side error as



Error when executing
query:org.apache.derby.client.am.DisconnectException: Execution failed
due to a distribution protocol error that caused deallocation of the
conversation. The requested command encountered an unarchitected and
implementation specific condition for which there was no architected
message.

com.ascential.xmeta.query.QueryException: Error when executing
query:org.apache.derby.client.am.DisconnectException: Execution failed
due to a distribution protocol error that caused deallocation of the
conversation. The requested command encountered an unarchitected and
implementation specific condition for which there was no architected
message.



I have used the IBM WebSphere Application Server Community Edition
Version 1.1 as the application server and the Derby 10.1 Net work server
as the database.



When create the datasource I have used the



<config-property-setting name="Password">pass</config-property-setting>

<config-property-setting
name="PortNumber">1527</config-property-setting>

<config-property-setting
name="ServerName">localhost</config-property-setting>

<config-property-setting
name="DatabaseName">TestDB</config-property-setting>

<config-property-setting
name="UserName">newuser</config-property-setting>



for the cloudscape datasource plan. Also I have used the

tranql-connector-derby-client-local-1.1.rar and
tranql-connector-derby-client-xa-1.1.rar as the tranql connectors.



While the above exception throws at the serverside following exception
will throw as derby.log

Server is ready to accept connections on port 1527.

Connection number: 1.

Connection number: 2.

Connection number: 3.

Connection number: 4.



------------ BEGIN SHUTDOWN ERROR STACK -------------



ERROR XSDG3: Meta-data for Container
***@67517852 could not be
accessed

at org.apache.derby.iapi.error.StandardException.newException(Unknown
Source)

at org.apache.derby.impl.store.raw.data.RAFContainer.run(Unknown Source)

at java.security.AccessController.doPrivileged1(Native Method)

at
java.security.AccessController.doPrivileged(AccessController.java(Compil
ed Code))

at
org.apache.derby.impl.store.raw.data.RAFContainer.openContainer(Unknown
Source)

at org.apache.derby.impl.store.raw.data.FileContainer.setIdent(Unknown
Source)

at org.apache.derby.impl.store.raw.data.RAFContainer.setIdentity(Unknown
Source)

at
org.apache.derby.impl.services.cache.CachedItem.takeOnIdentity(Unknown
Source)

at org.apache.derby.impl.services.cache.Clock.addEntry(Unknown Source)

at org.apache.derby.impl.services.cache.Clock.find(Unknown Source)

at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(U
nknown Source)

at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(U
nknown Source)

at org.apache.derby.impl.store.raw.xact.Xact.openContainer(Unknown
Source)

at
org.apache.derby.impl.store.access.btree.index.B2IFactory.readConglomera
te(Unknown Source)

at
org.apache.derby.impl.store.access.RAMAccessManager.conglomCacheFind(Unk
nown Source)

at
org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomera
te(Unknown Source)

at
org.apache.derby.impl.store.access.RAMTransaction.openStoreCost(Unknown
Source)

at
org.apache.derby.impl.sql.compile.CompilerContextImpl.getStoreCostContro
ller(Unknown Source)

at
org.apache.derby.impl.sql.compile.FromBaseTable.getStoreCostController(U
nknown Source)

at org.apache.derby.impl.sql.compile.FromBaseTable.estimateCost(Unknown
Source)

at
org.apache.derby.impl.sql.compile.OptimizerImpl.estimateTotalCost(Unknow
n Source)

at
org.apache.derby.impl.sql.compile.OptimizerImpl.costBasedCostOptimizable
(Unknown Source)

at
org.apache.derby.impl.sql.compile.OptimizerImpl.costOptimizable(Unknown
Source)



After the above exception happens any operation doing with my
application get fails.



Note: None of this issue is happening when my application is running
with

Derby 10.1 and IBM WebSphere Application Server 6.0.2.5.

So I have assumed this happens only when the Derby 10.1 and IBM
WebSphere Application Server Community Edition Version 1.1.



I am using the java version "1.4.2"

Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2)

Classic VM (build 1.4.2, J2RE 1.4.2 IBM Windows 32 build
cn142ifx-20051115 (SR3 + 94164 + 97403 + 97482) (JIT enabled: jitc))



This comes with the IBM Websphere IBM Websphere Application Server
6.0.2.5.



Even with the

java version "1.5.0"

Java(TM) 2 Runtime Environment, Standard Edition (build
pwi32devifx-20060818 (ifix: 108655: SR2 + 107350 + 108225 + 108262))

IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Windows XP x86-32
j9vmwi3223-20060504 (JIT enabled)

With comes with the IBM WebSphere Application Server Community Edition
Version 1.1 was not able to solve problem

Or when use a higher version of derby





I am using tranql-connector-derby-client-local-1.1.rar and
tranql-connector-derby-client-xa-1.1.rar as the tranql connectors not
the embedded one. Use the above TranQL is one of the project
requirements. Because of that I have not tested with embedded version of
the TranQL adapter.



The above exception happens when I run a set of test cases for my
application as a nightly build. The error occurs in the 3rd test class.
So I have removed the 3rd test class and run it again. I have found that
the same error happens in a different test class below. Each of the test
class has different functionalities





Is this a known issue among the community?

Is there any kind of solution for this?





Thanks,

Suraj

[Stanley Bradbury]

Post by Suraj Batuwana
Hi,
I have received a server side error as
Error when executing
query:org.apache.derby.client.am.DisconnectException: Execution failed
due to a distribution protocol error that caused deallocation of the
conversation. The requested command encountered an unarchitected and
implementation specific condition for which there was no architected
message.
com.ascential.xmeta.query.QueryException: Error when executing
query:org.apache.derby.client.am.DisconnectException: Execution failed
due to a distribution protocol error that caused deallocation of the
conversation. The requested command encountered an unarchitected and
implementation specific condition for which there was no architected
message.
I have used the IBM WebSphere Application Server Community Edition
Version 1.1 as the application server and the Derby 10.1 Net work
server as the database.
When create the datasource I have used the
<config-property-setting name="Password">pass</config-property-setting>
<config-property-setting name="PortNumber">1527</config-property-setting>
<config-property-setting
name="ServerName">localhost</config-property-setting>
<config-property-setting
name="DatabaseName">TestDB</config-property-setting>
<config-property-setting name="UserName">newuser</config-property-setting>
for the cloudscape datasource plan. Also I have used the
tranql-connector-derby-client-local-1.1.rar and
tranql-connector-derby-client-xa-1.1.rar as the tranql connectors.
While the above exception throws at the serverside following exception
will throw as derby.log
Server is ready to accept connections on port 1527.
Connection number: 1.
Connection number: 2.
Connection number: 3.
Connection number: 4.
------------ BEGIN SHUTDOWN ERROR STACK -------------
ERROR XSDG3: Meta-data for Container
be accessed
at org.apache.derby.iapi.error.StandardException.newException(Unknown
Source)
at org.apache.derby.impl.store.raw.data.RAFContainer.run(Unknown Source)
at java.security.AccessController.doPrivileged1(Native Method)
at
java.security.AccessController.doPrivileged(AccessController.java(Compiled
Code))
at
org.apache.derby.impl.store.raw.data.RAFContainer.openContainer(Unknown
Source)
at org.apache.derby.impl.store.raw.data.FileContainer.setIdent(Unknown
Source)
at
org.apache.derby.impl.store.raw.data.RAFContainer.setIdentity(Unknown
Source)
at
org.apache.derby.impl.services.cache.CachedItem.takeOnIdentity(Unknown
Source)
at org.apache.derby.impl.services.cache.Clock.addEntry(Unknown Source)
at org.apache.derby.impl.services.cache.Clock.find(Unknown Source)
at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown
Source)
at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown
Source)
at org.apache.derby.impl.store.raw.xact.Xact.openContainer(Unknown Source)
at
org.apache.derby.impl.store.access.btree.index.B2IFactory.readConglomerate(Unknown
Source)
at
org.apache.derby.impl.store.access.RAMAccessManager.conglomCacheFind(Unknown
Source)
at
org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown
Source)
at
org.apache.derby.impl.store.access.RAMTransaction.openStoreCost(Unknown
Source)
at
org.apache.derby.impl.sql.compile.CompilerContextImpl.getStoreCostController(Unknown
Source)
at
org.apache.derby.impl.sql.compile.FromBaseTable.getStoreCostController(Unknown
Source)
at
org.apache.derby.impl.sql.compile.FromBaseTable.estimateCost(Unknown
Source)
at
org.apache.derby.impl.sql.compile.OptimizerImpl.estimateTotalCost(Unknown
Source)
at
org.apache.derby.impl.sql.compile.OptimizerImpl.costBasedCostOptimizable(Unknown
Source)
at
org.apache.derby.impl.sql.compile.OptimizerImpl.costOptimizable(Unknown
Source)
After the above exception happens any operation doing with my
application get fails.
Note: None of this issue is happening when my application is running with
Derby 10.1 and IBM WebSphere Application Server 6.0.2.5.
So I have assumed this happens only when the Derby 10.1 and IBM
WebSphere Application Server Community Edition Version 1.1.
I am using the java version "1.4.2"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2)
Classic VM (build 1.4.2, J2RE 1.4.2 IBM Windows 32 build
cn142ifx-20051115 (SR3 + 94164 + 97403 + 97482) (JIT enabled: jitc))
This comes with the IBM Websphere IBM Websphere Application Server
6.0.2.5.
Even with the
java version "1.5.0"
Java(TM) 2 Runtime Environment, Standard Edition (build
pwi32devifx-20060818 (ifix: 108655: SR2 + 107350 + 108225 + 108262))
IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Windows XP x86-32
j9vmwi3223-20060504 (JIT enabled)
With comes with the IBM WebSphere Application Server Community Edition
Version 1.1 was not able to solve problem
Or when use a higher version of derby
I am using tranql-connector-derby-client-local-1.1.rar and
tranql-connector-derby-client-xa-1.1.rar as the tranql connectors not
the embedded one. Use the above TranQL is one of the project
requirements. Because of that I have not tested with embedded version
of the TranQL adapter.
The above exception happens when I run a set of test cases for my
application as a nightly build. The error occurs in the 3rd test
class. So I have removed the 3rd test class and run it again. I have
found that the same error happens in a different test class below.
Each of the test class has different functionalities
Is this a known issue among the community?
Is there any kind of solution for this?
Thanks,
Suraj

Hi Suraj -
Here's some possibilities:
Are you starting Network Server separately and then starting WAS CE (or
visa versa). If so the problem could be in how you are starting the
Network Server, the version of Network Server you are using or it could
be related to the fact that WAS CE starts it's own instance of Network
Server (as does the software it is based on, the Geronimo App Server).

If possible, use the Network Server instance that WAS CE starts and use
the client jarfile that comes with WAS CE. This will greatly simplify
the configuration and insure that the same version of Network Server and
Derby are used for all databases.

If you require a different version or separate instance of Network
Server be sure you start this instance listening on a different port
number that the one WAS CE uses AND be sure that the version of the
client jar matches or is a higher higher version than the most recent
version of the
Network Server being started (the client driver is backward compatible
but not necessarily forward compatible).

If this doesn't help please detail how you are starting Network Server
and supply the information from '....NetworkServerControl sysinfo' as
well as any console messages that may be displayed.
Is the derby.log file you cite found in the WAS CE installation tree or
elsewhere?

[Suraj Batuwana]

Hi Stanley,



My network server starts with jar files which come with derby using
cloudscape from the different port. The derby server starts with part of
the nightly build from maven as shown below.



<goal name="start-derby-server">

<attainGoal name="set-datasource-properties"/>

<j:thread file="thread.out">

<ant:java
classname="org.apache.derby.drda.NetworkServerControl" fork="yes">


<ant:classpath>

<ant:path
refid="maven.dependency.classpath"/>

<ant:pathelement
location="${maven.build.dest}"/>

</ant:classpath>

<ant:arg line="start -h localhost -p 1530"/>

</ant:java>

</j:thread>


</goal>



The default port of derby with comes with WASCE is 1527. Before starting
the derby I am creating a database in my maven target directory as

<goal name="init- derby -db">

<ant:java classname="org.apache.derby.tools.ij" fork="yes">


<ant:classpath>

<ant:path
refid="maven.dependency.classpath"/>

<ant:pathelement
location="${maven.build.dest}"/>

</ant:classpath>

<ant:arg line="-p
${maven.src.dir}/sql/ij.properties"/>

<ant:arg
line="${maven.src.dir}/sql//PersistentECore.sql"/>

</ant:java>

</goal>



ij.properties has the following values

ij.connection.testConn=jdbc:derby:target/testDB;create=true;user=testuse
r;bootPassword=testpass;



According to my application it is not possible to use the Network Server
instance that WAS CE starts.



Thanks,

Suraj

________________________________

From: Suraj Batuwana
Sent: Thursday, October 26, 2006 10:46 AM
To: 'derby-user-***@public.gmane.org'
Subject: Issue with Derby when using with IBM WebSphere Application
Server Community Edition



Hi,

I have received a server side error as



Error when executing
query:org.apache.derby.client.am.DisconnectException: Execution failed
due to a distribution protocol error that caused deallocation of the
conversation. The requested command encountered an unarchitected and
implementation specific condition for which there was no architected
message.

com.ascential.xmeta.query.QueryException: Error when executing
query:org.apache.derby.client.am.DisconnectException: Execution failed
due to a distribution protocol error that caused deallocation of the
conversation. The requested command encountered an unarchitected and
implementation specific condition for which there was no architected
message.



I have used the IBM WebSphere Application Server Community Edition
Version 1.1 as the application server and the Derby 10.1 Net work server
as the database.



When create the datasource I have used the



<config-property-setting name="Password">pass</config-property-setting>

<config-property-setting
name="PortNumber">1527</config-property-setting>

<config-property-setting
name="ServerName">localhost</config-property-setting>

<config-property-setting
name="DatabaseName">TestDB</config-property-setting>

<config-property-setting
name="UserName">newuser</config-property-setting>



for the cloudscape datasource plan. Also I have used the

tranql-connector-derby-client-local-1.1.rar and
tranql-connector-derby-client-xa-1.1.rar as the tranql connectors.



While the above exception throws at the serverside following exception
will throw as derby.log

Server is ready to accept connections on port 1527.

Connection number: 1.

Connection number: 2.

Connection number: 3.

Connection number: 4.



------------ BEGIN SHUTDOWN ERROR STACK -------------



ERROR XSDG3: Meta-data for Container
***@67517852 could not be
accessed

at org.apache.derby.iapi.error.StandardException.newException(Unknown
Source)

at org.apache.derby.impl.store.raw.data.RAFContainer.run(Unknown Source)

at java.security.AccessController.doPrivileged1(Native Method)

at
java.security.AccessController.doPrivileged(AccessController.java(Compil
ed Code))

at
org.apache.derby.impl.store.raw.data.RAFContainer.openContainer(Unknown
Source)

at org.apache.derby.impl.store.raw.data.FileContainer.setIdent(Unknown
Source)

at org.apache.derby.impl.store.raw.data.RAFContainer.setIdentity(Unknown
Source)

at
org.apache.derby.impl.services.cache.CachedItem.takeOnIdentity(Unknown
Source)

at org.apache.derby.impl.services.cache.Clock.addEntry(Unknown Source)

at org.apache.derby.impl.services.cache.Clock.find(Unknown Source)

at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(U
nknown Source)

at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(U
nknown Source)

at org.apache.derby.impl.store.raw.xact.Xact.openContainer(Unknown
Source)

at
org.apache.derby.impl.store.access.btree.index.B2IFactory.readConglomera
te(Unknown Source)

at
org.apache.derby.impl.store.access.RAMAccessManager.conglomCacheFind(Unk
nown Source)

at
org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomera
te(Unknown Source)

at
org.apache.derby.impl.store.access.RAMTransaction.openStoreCost(Unknown
Source)

at
org.apache.derby.impl.sql.compile.CompilerContextImpl.getStoreCostContro
ller(Unknown Source)

at
org.apache.derby.impl.sql.compile.FromBaseTable.getStoreCostController(U
nknown Source)

at org.apache.derby.impl.sql.compile.FromBaseTable.estimateCost(Unknown
Source)

at
org.apache.derby.impl.sql.compile.OptimizerImpl.estimateTotalCost(Unknow
n Source)

at
org.apache.derby.impl.sql.compile.OptimizerImpl.costBasedCostOptimizable
(Unknown Source)

at
org.apache.derby.impl.sql.compile.OptimizerImpl.costOptimizable(Unknown
Source)



After the above exception happens any operation doing with my
application get fails.



Note: None of this issue is happening when my application is running
with

Derby 10.1 and IBM WebSphere Application Server 6.0.2.5.

So I have assumed this happens only when the Derby 10.1 and IBM
WebSphere Application Server Community Edition Version 1.1.



I am using the java version "1.4.2"

Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2)

Classic VM (build 1.4.2, J2RE 1.4.2 IBM Windows 32 build
cn142ifx-20051115 (SR3 + 94164 + 97403 + 97482) (JIT enabled: jitc))



This comes with the IBM Websphere IBM Websphere Application Server
6.0.2.5.



Even with the

java version "1.5.0"

Java(TM) 2 Runtime Environment, Standard Edition (build
pwi32devifx-20060818 (ifix: 108655: SR2 + 107350 + 108225 + 108262))

IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Windows XP x86-32
j9vmwi3223-20060504 (JIT enabled)

With comes with the IBM WebSphere Application Server Community Edition
Version 1.1 was not able to solve problem

Or when use a higher version of derby





I am using tranql-connector-derby-client-local-1.1.rar and
tranql-connector-derby-client-xa-1.1.rar as the tranql connectors not
the embedded one. Use the above TranQL is one of the project
requirements. Because of that I have not tested with embedded version of
the TranQL adapter.



The above exception happens when I run a set of test cases for my
application as a nightly build. The error occurs in the 3rd test class.
So I have removed the 3rd test class and run it again. I have found that
the same error happens in a different test class below. Each of the test
class has different functionalities





Is this a known issue among the community?

Is there any kind of solution for this?





Thanks,

Suraj

[Stanley Bradbury]

Post by Suraj Batuwana
Hi Stanley,
My network server starts with jar files which come with derby using
cloudscape from the different port. The derby server starts with part
of the nightly build from maven as shown below.
<goal name="start-derby-server">
<attainGoal name="set-datasource-properties"/>
<j:thread file="thread.out">
<ant:java
classname="org.apache.derby.drda.NetworkServerControl" fork="yes">
<ant:classpath>
<ant:path
refid="maven.dependency.classpath"/>
<ant:pathelement
location="${maven.build.dest}"/>
</ant:classpath>
<ant:arg line="start -h localhost -p 1530"/>
</ant:java>
</j:thread>
</goal>
The default port of derby with comes with WASCE is 1527. Before
starting the derby I am creating a database in my maven target
directory as
<goal name="init- derby -db">
<ant:java classname="org.apache.derby.tools.ij"
fork="yes">
<ant:classpath>
<ant:path
refid="maven.dependency.classpath"/>
<ant:pathelement
location="${maven.build.dest}"/>
</ant:classpath>
<ant:arg line="-p
${maven.src.dir}/sql/ij.properties"/>
<ant:arg
line="${maven.src.dir}/sql//PersistentECore.sql"/>
</ant:java>
</goal>
ij.properties has the following values
ij.connection.testConn=jdbc:derby:target/testDB;create=true;user=testuser;bootPassword=testpass;
According to my application it is not possible to use the Network
Server instance that WAS CE starts.
Thanks,
Suraj
------------------------------------------------------------------------
*From:* Suraj Batuwana
*Sent:* Thursday, October 26, 2006 10:46 AM
*Subject:* Issue with Derby when using with IBM WebSphere Application
Server Community Edition
Hi,
I have received a server side error as
Error when executing
query:org.apache.derby.client.am.DisconnectException: Execution failed
due to a distribution protocol error that caused deallocation of the
conversation. The requested command encountered an unarchitected and
implementation specific condition for which there was no architected
message.
com.ascential.xmeta.query.QueryException: Error when executing
query:org.apache.derby.client.am.DisconnectException: Execution failed
due to a distribution protocol error that caused deallocation of the
conversation. The requested command encountered an unarchitected and
implementation specific condition for which there was no architected
message.
I have used the IBM WebSphere Application Server Community Edition
Version 1.1 as the application server and the Derby 10.1 Net work
server as the database.
When create the datasource I have used the
<config-property-setting name="Password">pass</config-property-setting>
<config-property-setting name="PortNumber">1527</config-property-setting>
<config-property-setting
name="ServerName">localhost</config-property-setting>
<config-property-setting
name="DatabaseName">TestDB</config-property-setting>
<config-property-setting name="UserName">newuser</config-property-setting>
for the cloudscape datasource plan. Also I have used the
tranql-connector-derby-client-local-1.1.rar and
tranql-connector-derby-client-xa-1.1.rar as the tranql connectors.
While the above exception throws at the serverside following exception
will throw as derby.log
Server is ready to accept connections on port 1527.
Connection number: 1.
Connection number: 2.
Connection number: 3.
Connection number: 4.
------------ BEGIN SHUTDOWN ERROR STACK -------------
ERROR XSDG3: Meta-data for Container
be accessed

==== SNIP =====

Hi Suraj -

Based on your response I am assuming that this part of my response
describes the change that needs to be made:

"...AND be sure that the version of the client jar matches or is a
higher higher version than the most recent version of the
Network Server being started (the client driver is backward compatible
but not necessarily forward compatible). ..."

An incompatibility between the clientDriver and the Network Server would
result in a Distributed Protocol Error.

My guess is that the connection that is failing (the datasource defined
in WAS CE) is using an older version of derbyclient.jar than is
supported for the version of derby you downloaded. If you verify this
to be the case then shutdown both systems, overwrite the earlier jarfile
with the more recent version, restart and I think all will be well.

In addition, if my assumptions about your configuration are correct, the
datasource you define for WAS CE is trying to connecting to the WAS CE
network server (e.g. localhost:1527) and trying to access the database
of the Maven Network Server - you should be addressing the Maven Network
Server ( <ant:arg line="start -h localhost -p 1530"/>) - this could
result in the "ERROR XSDG3: Meta-data for Container.." error you as
seeing (presumably in the derby.log for WAS CE, the database might
already be locked by the Maven Network Server).

It is a shame that WAS CE does allow you to utilize the Network Server
it starts, I assume this is done because of security concerns.

If this doesn't help please ... supply the information from
'....NetworkServerControl sysinfo' as well as any console messages that
may be displayed. (am referring to the Network Server console messages
for both the WAS CE instance and one started by Maven).

HTH

[Suresh Thalamati]

Post by Stanley Bradbury

Post by Suraj Batuwana
Hi Stanley,

In addition, if my assumptions about your configuration are correct, the
datasource you define for WAS CE is trying to connecting to the WAS CE
network server (e.g. localhost:1527) and trying to access the database
of the Maven Network Server - you should be addressing the Maven Network
Server ( <ant:arg line="start -h localhost -p 1530"/>) - this could
result in the "ERROR XSDG3: Meta-data for Container.." error you as
seeing (presumably in the derby.log for WAS CE, the database might
already be locked by the Maven Network Server).
It is a shame that WAS CE does allow you to utilize the Network Server
it starts, I assume this is done because of security concerns.
If this doesn't help please ... supply the information from
'....NetworkServerControl sysinfo' as well as any console messages that
may be displayed. (am referring to the Network Server console messages
for both the WAS CE instance and one started by Maven).

Hi Stan and Suraj.

I am bit puzzled about "ERROR XSDG3: Meta-data for Container.." and
would like to learn, why this is happening in this setup. Typically
this error occurs, if the container files are corrupted (due to disk
corruption or the database is on some kind of network file system
where file syncs are not getting synced.


Suraj,

1) is this error happening only on a particular JVM ? ( I saw in
your e-mail you were using 1.5(j9) and 1.4 ; On which jvm you saw the
error first ? If once the corruption occurs , it won't go away, so it
does not matter if you change the jvm, unless you created new
database , for both runs.


2) are you creating the database on a network drive ?


3) Are you seeing any files with zero length in the <database name>
seg0 directory of the database.


4) Are you able to reproduce this error consistently on a new database ?




Thanks
-suresh

[Suraj Batuwana]

Hi Suresh,

1. I have tested on 1.5 (j9) to check the same error is happening there. And
it does. So the error is not targeted a perticuler JVM. Basically I am using
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2)
Classic VM (build 1.4.2, J2RE 1.4.2 IBM Windows 32 build cn142ifx-20051115
(SR3 + 94164 + 97403 + 97482) (JIT enabled: jitc))

To WASCE(server side) and to run maven(client side)

2. I am creating the database on the local drive. Not in a network drive

3.I do not see any files with zero length in the <database name>
seg0 directory of the database.

4) I am able to reproduce this error consistently on a new database also.

5) I have attached my derby.log for your reference.

Looking at that I have found several interesting things.(I am assuming)
While the nightly build tests are happing with derby/WASCE the derby shuting
down and booting its instances. It seems that shuting down of instances are
happening while the live connections are still there.

But when I running the same code base with derby/websphere 6.0.2.5 only one
time boot up is happend. here It tries to shutdown and boot up the instance
several times. Is there a way to stop or controll the shuting down and boot
up the derby instance.

I also found that error is commong from error message coming from the
derby.org.apache.derby.client.net.NetConnectionReply a class which is in the
derbyclient.jar


-Suraj

Post by Suresh Thalamati

Post by Stanley Bradbury

Post by Suraj Batuwana
Hi Stanley,

In addition, if my assumptions about your configuration are correct, the
datasource you define for WAS CE is trying to connecting to the WAS CE
network server (e.g. localhost:1527) and trying to access the database
of the Maven Network Server - you should be addressing the Maven Network
Server ( <ant:arg line="start -h localhost -p 1530"/>) - this could
result in the "ERROR XSDG3: Meta-data for Container.." error you as
seeing (presumably in the derby.log for WAS CE, the database might
already be locked by the Maven Network Server).
It is a shame that WAS CE does allow you to utilize the Network Server
it starts, I assume this is done because of security concerns.
If this doesn't help please ... supply the information from
'....NetworkServerControl sysinfo' as well as any console messages that
may be displayed. (am referring to the Network Server console messages
for both the WAS CE instance and one started by Maven).

Hi Stan and Suraj.
I am bit puzzled about "ERROR XSDG3: Meta-data for Container.." and
would like to learn, why this is happening in this setup. Typically
this error occurs, if the container files are corrupted (due to disk
corruption or the database is on some kind of network file system
where file syncs are not getting synced.
Suraj,
1) is this error happening only on a particular JVM ? ( I saw in
your e-mail you were using 1.5(j9) and 1.4 ; On which jvm you saw the
error first ? If once the corruption occurs , it won't go away, so it
does not matter if you change the jvm, unless you created new
database , for both runs.
2) are you creating the database on a network drive ?
3) Are you seeing any files with zero length in the <database name>
seg0 directory of the database.
4) Are you able to reproduce this error consistently on a new database ?
Thanks
-suresh

http://www.nabble.com/file/3919/derby.log derby.log

[Suresh Thalamati]

Post by Suraj Batuwana
Hi Suresh,
3.I do not see any files with zero length in the <database name>
seg0 directory of the database.

That's good news. zero-length files are the real scary ones.

Post by Suraj Batuwana
4) I am able to reproduce this error consistently on a new database also.
5) I have attached my derby.log for your reference.
Looking at that I have found several interesting things.(I am assuming)
While the nightly build tests are happing with derby/WASCE the derby shuting
down and booting its instances. It seems that shuting down of instances are
happening while the live connections are still there.
But when I running the same code base with derby/websphere 6.0.2.5 only one
time boot up is happend. here It tries to shutdown and boot up the instance
several times. Is there a way to stop or controll the shuting down and boot
up the derby instance.

Thanks for providing more info on this problem. By looking at the
derby.log , the reason you are seeing many boot/shutdown sequences is
when derby fails to open the container file, it is serious error , so
it shutdown the database automatically to avoid any chance for
corruptions. When a next attempt was made to connect to the database
,it is attempting to boot again.


I looked at the stacks in the derby.log. Real problem seems to be for
some reason , derby engine is not unable to open the container files.
It is happening while compiling a query and also during boot on
recovery. That makes me inclined to believe it is an environment issue.

D:\wasce_build\TestBuild\build1\target\sampledb\seg0\c5d1.dat (Cannot
create a file when that file already exists) at
java.io.RandomAccessFile.open(Native Method) at
java.io.RandomAccessFile.<INIT>(RandomAccessFile.java(Compiled Code))

From the derby.log : unable to open these two files seg0\c5d1.dat and
seg0\c2d0.dat.

Please check the permissions/owner ship of the file in the seg0.
Typically the reason will be "Access denied ....". So I doubt
that is the case.

I wrote a small program to find what causes "Cannot create a file
when that file already exists" error case, only time I am able to hit
that is when system ran out of file handles(file descriptors).

After bit of googling found a reporred bug on jdk142
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4189011, may be
you are hitting this one.

When I tried my small program on jdk15, it managed to open 10000
files, whereas on jdk142 (both on ibm and sun) it was failing
around 2020 .

Derby does need so many file handles, it might be that something else
is leaking the file descriptors and derby is failing as side-effect.
It might be worth finding, all the open file handles on the system
when this error occurs. I don't know how to do that on Windows.

One another thing, I would try is after you hit the error. Shutdown
WAS and the network server and then connect to the database using IJ
and see if you can perform some queries on the database. If this works
then is most likely a file descriptors problem.

Thanks
-suresh

[Suraj Batuwana]

Hi Suresh,
I have able to connect to the connect to the database using IJ
and performed some queries on the database. Is that means it has a file
descriptors problem ?
how do we over come such an issue ?

While I am testing for derby/WASCE i got another requirement, that is to
deploy my application in
WAS6.0.2.5/derby10.2.1.6. But this time the requirement was to use the
embedded driver.

So I have used the org.apache.derby.jdbc.EmbeddedDriver as the jdbc driver
and jdbc:derby:TestDB as
the database URL

This time I got a different error from the client side test classes
Can't connect to url=jdbc:derby:TestDB
java.sql.SQLException: Failed to start database 'TestDB', see the next
exception for details.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown
Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.newSQLException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source)
at org.apache.derby.jdbc.Driver30.getNewEmbedConnection(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)

from the server side error was
Caused by: java.sql.SQLException: Failed to start database
'E:\source\database\TestDB', see the next exception for details.DSRA0010E:
SQL State = XJ040, Error Code = 40,000DSRA0010E: SQL State = XJ040, Error
Code = 40,000
at sun.reflect.GeneratedConstructorAccessor498.newInstance(Unknown Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java(Compiled
Code))
at java.lang.reflect.Constructor.newInstance(Constructor.java(Compiled
Code))
at
com.ibm.websphere.rsadapter.GenericDataStoreHelper.mapExceptionHelper(GenericDataStoreHelper.java:501)
at
com.ibm.websphere.rsadapter.GenericDataStoreHelper.mapException(GenericDataStoreHelper.java:544)
at
com.ibm.ws.rsadapter.spi.WSRdbDataSource.getPooledConnection(WSRdbDataSource.java:1037)

I have used the follings to create websphere derby datasource
implementing class
"org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource"
implementing class for XA "org.apache.derby.jdbc.EmbeddedXADataSource"
dataSource Helper Class Name
"com.ibm.websphere.rsadapter.DerbyDataStoreHelper"

I have test this with ibm jdk 1.4.2. Is this happening due to the same kind
of a issue as you described ?
But this time i did not get any errors on derby.log

Thanks,
Suraj

Post by Suresh Thalamati

Post by Suraj Batuwana
Hi Suresh,
3.I do not see any files with zero length in the <database name>
seg0 directory of the database.

That's good news. zero-length files are the real scary ones.

Post by Suraj Batuwana
4) I am able to reproduce this error consistently on a new database also.
5) I have attached my derby.log for your reference.
Looking at that I have found several interesting things.(I am assuming)
While the nightly build tests are happing with derby/WASCE the derby shuting
down and booting its instances. It seems that shuting down of instances are
happening while the live connections are still there.
But when I running the same code base with derby/websphere 6.0.2.5 only one
time boot up is happend. here It tries to shutdown and boot up the instance
several times. Is there a way to stop or controll the shuting down and boot
up the derby instance.

Thanks for providing more info on this problem. By looking at the
derby.log , the reason you are seeing many boot/shutdown sequences is
when derby fails to open the container file, it is serious error , so
it shutdown the database automatically to avoid any chance for
corruptions. When a next attempt was made to connect to the database
,it is attempting to boot again.
I looked at the stacks in the derby.log. Real problem seems to be for
some reason , derby engine is not unable to open the container files.
It is happening while compiling a query and also during boot on
recovery. That makes me inclined to believe it is an environment issue.
D:\wasce_build\TestBuild\build1\target\sampledb\seg0\c5d1.dat (Cannot
create a file when that file already exists) at
java.io.RandomAccessFile.open(Native Method) at
java.io.RandomAccessFile.<INIT>(RandomAccessFile.java(Compiled Code))
From the derby.log : unable to open these two files seg0\c5d1.dat and
seg0\c2d0.dat.
Please check the permissions/owner ship of the file in the seg0.
Typically the reason will be "Access denied ....". So I doubt
that is the case.
I wrote a small program to find what causes "Cannot create a file
when that file already exists" error case, only time I am able to hit
that is when system ran out of file handles(file descriptors).
After bit of googling found a reporred bug on jdk142
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4189011, may be
you are hitting this one.
When I tried my small program on jdk15, it managed to open 10000
files, whereas on jdk142 (both on ibm and sun) it was failing
around 2020 .
Derby does need so many file handles, it might be that something else
is leaking the file descriptors and derby is failing as side-effect.
It might be worth finding, all the open file handles on the system
when this error occurs. I don't know how to do that on Windows.
One another thing, I would try is after you hit the error. Shutdown
WAS and the network server and then connect to the database using IJ
and see if you can perform some queries on the database. If this works
then is most likely a file descriptors problem.
Thanks
-suresh

[Suresh Thalamati]

Post by Suraj Batuwana
Hi Suresh,
I have able to connect to the connect to the database using IJ
and performed some queries on the database. Is that means it has a file
descriptors problem ?

Thanks a lot for verifying. If you are able to boot , the it is not a
corruption issue, for sure. From the error, as I mentioned earlier
my suspicion is file descriptors are leaking some where in your
application.

Post by Suraj Batuwana
how do we over come such an issue ?

If you can not find what part of the application is leaking the
descriptors, I have no idea, how to solve it.

Post by Suraj Batuwana
While I am testing for derby/WASCE i got another requirement, that is to
deploy my application in
WAS6.0.2.5/derby10.2.1.6. But this time the requirement was to use the
embedded driver.
So I have used the org.apache.derby.jdbc.EmbeddedDriver as the jdbc driver
and jdbc:derby:TestDB as
the database URL
This time I got a different error from the client side test classes
Can't connect to url=jdbc:derby:TestDB
java.sql.SQLException: Failed to start database 'TestDB', see the next

<snip ...>

Post by Suraj Batuwana
I have test this with ibm jdk 1.4.2. Is this happening due to the same kind
of a issue as you described ?
But this time i did not get any errors on derby.log

If you are not seeing any errors in derby.log, then may be it is a
different problem. It might be just a setup issue, I have no
knowledge of setting up embedded database in the Webspehere
environment, may be others on the list will be able to help you.


Thanks
-suresh