Bug #57
db2ldap : Binary attribute seems to be corrupted
| Status: | Closed | Start: | 02/06/2009 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assigned to: | Jonathan Clarke | % Done: | 0% |
|
| Category: | Core | |||
| Target version: | 1.1.0 | |||
| Problem in version: |
Description
I am in the process of syncing a MySQL database with an openldap directory.
In the MySQL table, there is a blob containing a JPEG picture of the person.
The SQL field is mapped directly to the jpegPhoto LDAP attribute like that:
<result property="jpegPhoto" column="photo"/>
The synchronization process is working fine except that the JPEG in the LDAP directory seems to be invalid.
An export of the field is not a valid JPEG file and no LDAP browser displays the picture correctly.
It seems that binary attributes should be handled differently than others. See that forum post for information about this.
The parameter "java.naming.ldap.attributes.binary" should contain a list of LDAP attributes to be handled as binary.
Associated revisions
Add a full test for ldap2ldap sync and clean. Refs #57.
References #57. Handle binary attributes in BeanComparator (thanks to Jérôme Schell). Various cleanups.
History
Updated by Jérôme Schell about 1 year ago
After some investigation, I'm progressing ;)
I am now setting the jpegPhoto attribute as byte[] in the flat object class like that:
private byte[] jpegPhoto;
public final byte[] getJpegPhoto() {
return jpegPhoto;
}
public final void setJpegPhoto(final byte[] value) {
this.jpegPhoto = value;
}
The jpegPhoto attribute is of type List<byte[]> in InetOrgPerson class:
private List<byte[]> jpegPhoto;
public final List<byte[]> getJpegPhoto() {
return jpegPhoto;
}
public final void setJpegPhoto(final List<byte[]> values) {
jpegPhoto = values;
}
With that setting, the jpegPhoto attribute is not setted in the bean. The problem is in the method setUpFromObject of the org.lsc.objects.top class. The type byte[] is not handled at all.
I modified the method like that (brute force modification :) ):
--- lsc-core-1.1-SNAPSHOT/src/main/java/org/lsc/objects/top.java 2009-06-04 08:58:19.000000000 +0200
+++ lsc-core/src/main/java/org/lsc/objects/top.java 2009-06-04 10:00:06.586891780 +0200
@@ -127,35 +127,28 @@
// on a empty value ?
// localMethod.invoke(this, new Object[] {});
LOGGER.debug("No need to call a method with an empty value ... (" + paramName + ")");
- } else if (returnType == String.class) {
- if (toReturnTypes != null && toReturnTypes[0] == String.class) {
- localMethod.invoke(this, new Object[] { returnedObject });
- } else if (toReturnTypes != null && toReturnTypes[0] == List.class) {
- paramsToUse = new ArrayList<Object>();
- paramsToUse.add(returnedObject);
- localMethod.invoke(this, new Object[] { paramsToUse });
- } else {
- LOGGER.error("Unable to manage translation from String to " + toReturnTypes[0] + " for " + paramName+ "!");
- }
- } else if (returnType == Date.class) {
- if (toReturnTypes != null && toReturnTypes[0] == Date.class) {
- localMethod.invoke(this, new Object[] { returnedObject });
- } else if (toReturnTypes != null && toReturnTypes[0] == List.class) {
- paramsToUse = new ArrayList<Object>();
- paramsToUse.add(returnedObject);
- localMethod.invoke(this, new Object[] { paramsToUse });
- } else {
- LOGGER.error("Unable to manage translation from Date to " + toReturnTypes[0] + " for " + paramName+ "!");
- }
- } else if (returnType == List.class) {
- if (toReturnTypes != null && toReturnTypes[0] == String.class) {
- LOGGER.error("Unable to manage translation from List to String for " + paramName+ "!");
- } else if (toReturnTypes != null && toReturnTypes[0] == List.class) {
- LOGGER.debug("Method invocation : " + localMethod.getName());
- localMethod.invoke(this, new Object[] { returnedObject });
- } else {
- LOGGER.error("Unable to manage translation from List to " + toReturnTypes[0] + " for " + paramName+ "!");
- }
+ } else {
+ if (toReturnTypes != null && toReturnTypes[0] == List.class) {
+ LOGGER.debug("Method invocation: " + localMethod.getName());
+ try {
+ paramsToUse = new ArrayList<Object>();
+ paramsToUse.add(returnedObject);
+ localMethod.invoke(this, new Object[] { paramsToUse });
+ } catch (IllegalArgumentException e) {
+ LOGGER.error("Bad argument invoking " + localMethod.getName() + " for attribute " + paramName);
+ e.printStackTrace();
+ }
+ } else if (toReturnTypes != null && toReturnTypes[0] == returnType) {
+ LOGGER.debug("Method invocation: " + localMethod.getName());
+ try {
+ localMethod.invoke(this, new Object[] { returnedObject });
+ } catch (IllegalArgumentException e) {
+ LOGGER.error("Bad argument invoking " + localMethod.getName() + " for attribute " + paramName);
+ e.printStackTrace();
+ }
+ } else {
+ LOGGER.error("Unable to manage translation from " + returnType + " to " + toReturnTypes[0] + " for " + paramName+ "!");
+ }
}
} else {
if (paramName.compareToIgnoreCase("class") != 0) {
Now the attribute is well setted in the bean (with a good size).
Nevertheless the attribute inserted into the destination LDAP does not have the right size...
According to this the jpegPhoto is automatically handled as binary by JNDI. I can't for now understand why my final value is bigger in size than the original one...
Updated by Jérôme Schell about 1 year ago
Ok, I think I've got it ;)
The corruption appears in the method compareAttribute of the BeanComparator class:
} else if (o.getClass().isAssignableFrom(byte[].class)) {
dstAttrValues.add(new String((byte[]) o));
The use of "new String((byte[]) o)" seems to kill the binary attributes.
By aggressively replacing:
List<String> dstAttrValues = new ArrayList<String>(dstAttr.size());
by
List<Object> dstAttrValues = new ArrayList<Object>(dstAttr.size());
I managed to successfully synchronize the jpegPhoto attribute \o/
I think the problem will be the same in the mergeAttributes method.
Any idea on how to fix that?
The problem with my fix is that jpegPhoto attribute is always considered as being modified so it seems the byte[] comparison should be a special case...
Updated by Jonathan Clarke about 1 year ago
Jérôme Schell wrote:
Ok, I think I've got it ;)
The corruption appears in the method compareAttribute of the BeanComparator class: [...]
The use of "new String((byte[]) o)" seems to kill the binary attributes.
Aha. I remember this - it was a fix for issue #45... which needed to read userPassword (which is binary syntax) as a String.
I guess we need some kind of parameter to specify how to treat each attribute - ie "this is real binary, use byte[]" or "this should be a String, do new String(the attribute value)". Comments ?
I have the feeling that bug #55 may be related to this as well...
Updated by Jérôme Schell about 1 year ago
- File lsc-core-binary-attribute.diff added
The patch attached seems to work for me.
I didn't test attribute merging...
Updated by Jérôme Schell about 1 year ago
- File lsc-core-binary-attribute-2.diff added
Another patch to be able to use byte[] attributes in db2sql mode.
To be examined carefully as I simplified a lot the code (certainly too much...).
Updated by Jonathan Clarke about 1 year ago
Hi Jérôme,
I had a look at your patches. Thanks for posting them.
The first one, in BeanComparator, seems pretty clear - you're handling all binary attributes specially, and removing the new String(<binary value>). This looks OK, except for one thing I'm unsure about : will bug #45 reappear? We should really write a test for that.
About the second one, I am unclear as to what it's trying to do. Could you provide some explanation, maybe some comments in the code or a description in this issue?
Thanks in advance,
Jon
Updated by Jérôme Schell about 1 year ago
Jonathan Clarke wrote:
Hi Jérôme,
I had a look at your patches. Thanks for posting them.
The first one, in BeanComparator, seems pretty clear - you're handling all binary attributes specially, and removing the new String(<binary value>). This looks OK, except for one thing I'm unsure about : will bug #45 reappear? We should really write a test for that.
Huum, I don't think #45 will reappear as you are now handling the binary case in AbstractBean.java. Nevertheless I don't know if you should cast every values to String here, as some can now be byte[]...
About the second one, I am unclear as to what it's trying to do. Could you provide some explanation, maybe some comments in the code or a description in this issue?
As stated in my first post on this issue, the method setUpFromObject of the org.lsc.objects.top class doesn't handle the case when attributes are of type byte[] so these attributes are not imported into the final bean. That's why I just tried in my patch to "generalize" the way attributes are filled in the bean.
Is it more clear ? ;)
Updated by Jonathan Clarke about 1 year ago
- Status changed from New to Feedback
- Assigned to set to Jonathan Clarke
- Target version set to 1.1.0
Hi Jérôme,
I've tested your patches, using both the new Ldap2LdapSyncTest, and the quickstart tutorial. All in all, they work great! :)
I have applied them, with some modifications. For information:- In BeanComparator, method "compareAttributes": check that both source and dest Object is a byte[], not just one. This caused a cast error.
- In top, switch around the two parts of the if, to ensure that if the get*() method returns the same as our set*() method, we just call it.
- Various cleanups in BeanComparator.
- Indentation :)
Can you test, please, and confirm that this works the way you want it?
Thanks a lot for your testing, and diving into the code to make these patches.
Updated by Jérôme Schell about 1 year ago
Jonathan Clarke wrote:
Can you test, please, and confirm that this works the way you want it?
I've just tested the new core revision in my ldap2ldap and db2ldap setup. All seems to work fine. userPassword and jpegPhoto attributes are ok. Creation and comparison are handled fine.
Thanks Jonathan.
Updated by Jonathan Clarke about 1 year ago
- Status changed from Feedback to Closed
Great! Thanks for the feedback. I'm closing this issue as it's resolved now.