Bug #57

db2ldap : Binary attribute seems to be corrupted

Added by Jérôme Schell about 1 year ago. Updated about 1 year ago.

Status:Closed Start:02/06/2009
Priority:Normal Due date:
Assigned to:Jonathan Clarke % Done:

0%

Category:Core
Target version:1.1.0
Problem in version:

Description

I am in the process of syncing a MySQL database with an openldap directory.
In the MySQL table, there is a blob containing a JPEG picture of the person.
The SQL field is mapped directly to the jpegPhoto LDAP attribute like that:

<result property="jpegPhoto" column="photo"/>

The synchronization process is working fine except that the JPEG in the LDAP directory seems to be invalid.
An export of the field is not a valid JPEG file and no LDAP browser displays the picture correctly.

It seems that binary attributes should be handled differently than others. See that forum post for information about this.
The parameter "java.naming.ldap.attributes.binary" should contain a list of LDAP attributes to be handled as binary.

lsc-core-binary-attribute.diff - Binary attribute handling patch in BeanComparator.java (7.6 KB) Jérôme Schell, 04/06/2009 15:39

lsc-core-binary-attribute-2.diff - byte[] attributes handling patch in top.java (2.9 KB) Jérôme Schell, 04/06/2009 15:44


Related issues

related to Bug #45: Reading userPassword or other binary syntax attributes causes an error Closed 27/04/2009
related to Bug #110: userPassword is always modified Closed 16/07/2009

Associated revisions

Revision 265
Added by Jonathan Clarke about 1 year ago

Add a full test for ldap2ldap sync and clean. Refs #57.

Revision 277
Added by Jonathan Clarke about 1 year ago

References #57. Handle binary attributes in BeanComparator (thanks to Jérôme Schell). Various cleanups.

History

Updated by Jérôme Schell about 1 year ago

After some investigation, I'm progressing ;)

I am now setting the jpegPhoto attribute as byte[] in the flat object class like that:

  private byte[] jpegPhoto;

  public final byte[] getJpegPhoto() {
    return jpegPhoto;
  }

  public final void setJpegPhoto(final byte[] value) {
    this.jpegPhoto = value;
  }

The jpegPhoto attribute is of type List<byte[]> in InetOrgPerson class:

  private List<byte[]> jpegPhoto;

  public final List<byte[]> getJpegPhoto() {
    return jpegPhoto;
  }

  public final void setJpegPhoto(final List<byte[]> values) {
    jpegPhoto = values;
  }

With that setting, the jpegPhoto attribute is not setted in the bean. The problem is in the method setUpFromObject of the org.lsc.objects.top class. The type byte[] is not handled at all.
I modified the method like that (brute force modification :) ):

--- lsc-core-1.1-SNAPSHOT/src/main/java/org/lsc/objects/top.java    2009-06-04 08:58:19.000000000 +0200
+++ lsc-core/src/main/java/org/lsc/objects/top.java    2009-06-04 10:00:06.586891780 +0200
@@ -127,35 +127,28 @@
                         // on a empty value ?
                         // localMethod.invoke(this, new Object[] {});
                         LOGGER.debug("No need to call a method with an empty value ... (" + paramName + ")");
-                    } else if (returnType == String.class) {
-                        if (toReturnTypes != null && toReturnTypes[0] == String.class) {
-                            localMethod.invoke(this, new Object[] { returnedObject });
-                        } else if (toReturnTypes != null && toReturnTypes[0] == List.class) {
-                            paramsToUse = new ArrayList<Object>();
-                            paramsToUse.add(returnedObject);
-                            localMethod.invoke(this, new Object[] { paramsToUse });
-                        } else {
-                            LOGGER.error("Unable to manage translation from String to "    + toReturnTypes[0] + " for " + paramName+ "!");
-                        }
-                    } else if (returnType == Date.class) {
-                        if (toReturnTypes != null && toReturnTypes[0] == Date.class) {
-                            localMethod.invoke(this, new Object[] { returnedObject });
-                        } else if (toReturnTypes != null && toReturnTypes[0] == List.class) {
-                            paramsToUse = new ArrayList<Object>();
-                            paramsToUse.add(returnedObject);
-                            localMethod.invoke(this, new Object[] { paramsToUse });
-                        } else {
-                            LOGGER.error("Unable to manage translation from Date to " + toReturnTypes[0] + " for " + paramName+ "!");
-                        }
-                    } else if (returnType == List.class) {
-                        if (toReturnTypes != null && toReturnTypes[0] == String.class) {
-                            LOGGER.error("Unable to manage translation from List to String for " + paramName+ "!");
-                        } else if (toReturnTypes != null && toReturnTypes[0] == List.class) {
-                            LOGGER.debug("Method invocation : " + localMethod.getName());
-                            localMethod.invoke(this, new Object[] { returnedObject });
-                        } else {
-                            LOGGER.error("Unable to manage translation from List to " + toReturnTypes[0] + " for " + paramName+ "!");
-                        }
+                    } else {
+                      if (toReturnTypes != null && toReturnTypes[0] == List.class) {
+              LOGGER.debug("Method invocation: " + localMethod.getName());
+              try {
+                paramsToUse = new ArrayList<Object>();
+                paramsToUse.add(returnedObject);
+                localMethod.invoke(this, new Object[] { paramsToUse });
+              } catch (IllegalArgumentException e) {
+                LOGGER.error("Bad argument invoking " + localMethod.getName() + " for attribute " + paramName);
+                e.printStackTrace();
+              }
+                      } else if (toReturnTypes != null && toReturnTypes[0] == returnType) {
+                        LOGGER.debug("Method invocation: " + localMethod.getName());
+                        try {
+                          localMethod.invoke(this, new Object[] { returnedObject });
+                        } catch (IllegalArgumentException e) {
+                          LOGGER.error("Bad argument invoking " + localMethod.getName() + " for attribute " + paramName);
+                          e.printStackTrace();
+                        }
+            } else {
+              LOGGER.error("Unable to manage translation from " + returnType + " to " + toReturnTypes[0] + " for " + paramName+ "!");
+            }
                     }
                 } else {
                     if (paramName.compareToIgnoreCase("class") != 0) {

Now the attribute is well setted in the bean (with a good size).
Nevertheless the attribute inserted into the destination LDAP does not have the right size...

According to this the jpegPhoto is automatically handled as binary by JNDI. I can't for now understand why my final value is bigger in size than the original one...

Updated by Jérôme Schell about 1 year ago

Ok, I think I've got it ;)

The corruption appears in the method compareAttribute of the BeanComparator class:

            } else if (o.getClass().isAssignableFrom(byte[].class)) {
                dstAttrValues.add(new String((byte[]) o));

The use of "new String((byte[]) o)" seems to kill the binary attributes.

By aggressively replacing:

  List<String> dstAttrValues = new ArrayList<String>(dstAttr.size());

by
  List<Object> dstAttrValues = new ArrayList<Object>(dstAttr.size());

I managed to successfully synchronize the jpegPhoto attribute \o/
I think the problem will be the same in the mergeAttributes method.

Any idea on how to fix that?
The problem with my fix is that jpegPhoto attribute is always considered as being modified so it seems the byte[] comparison should be a special case...

Updated by Jonathan Clarke about 1 year ago

Jérôme Schell wrote:

Ok, I think I've got it ;)

The corruption appears in the method compareAttribute of the BeanComparator class: [...]

The use of "new String((byte[]) o)" seems to kill the binary attributes.

Aha. I remember this - it was a fix for issue #45... which needed to read userPassword (which is binary syntax) as a String.

I guess we need some kind of parameter to specify how to treat each attribute - ie "this is real binary, use byte[]" or "this should be a String, do new String(the attribute value)". Comments ?

I have the feeling that bug #55 may be related to this as well...

Updated by Jérôme Schell about 1 year ago

The patch attached seems to work for me.
I didn't test attribute merging...

Updated by Jérôme Schell about 1 year ago

Another patch to be able to use byte[] attributes in db2sql mode.
To be examined carefully as I simplified a lot the code (certainly too much...).

Updated by Jonathan Clarke about 1 year ago

Hi Jérôme,

I had a look at your patches. Thanks for posting them.

The first one, in BeanComparator, seems pretty clear - you're handling all binary attributes specially, and removing the new String(<binary value>). This looks OK, except for one thing I'm unsure about : will bug #45 reappear? We should really write a test for that.

About the second one, I am unclear as to what it's trying to do. Could you provide some explanation, maybe some comments in the code or a description in this issue?

Thanks in advance,
Jon

Updated by Jérôme Schell about 1 year ago

Jonathan Clarke wrote:

Hi Jérôme,

I had a look at your patches. Thanks for posting them.

The first one, in BeanComparator, seems pretty clear - you're handling all binary attributes specially, and removing the new String(<binary value>). This looks OK, except for one thing I'm unsure about : will bug #45 reappear? We should really write a test for that.

Huum, I don't think #45 will reappear as you are now handling the binary case in AbstractBean.java. Nevertheless I don't know if you should cast every values to String here, as some can now be byte[]...

About the second one, I am unclear as to what it's trying to do. Could you provide some explanation, maybe some comments in the code or a description in this issue?

As stated in my first post on this issue, the method setUpFromObject of the org.lsc.objects.top class doesn't handle the case when attributes are of type byte[] so these attributes are not imported into the final bean. That's why I just tried in my patch to "generalize" the way attributes are filled in the bean.
Is it more clear ? ;)

Updated by Jonathan Clarke about 1 year ago

  • Status changed from New to Feedback
  • Assigned to set to Jonathan Clarke
  • Target version set to 1.1.0

Hi Jérôme,

I've tested your patches, using both the new Ldap2LdapSyncTest, and the quickstart tutorial. All in all, they work great! :)

I have applied them, with some modifications. For information:
  • In BeanComparator, method "compareAttributes": check that both source and dest Object is a byte[], not just one. This caused a cast error.
  • In top, switch around the two parts of the if, to ensure that if the get*() method returns the same as our set*() method, we just call it.
  • Various cleanups in BeanComparator.
  • Indentation :)

Can you test, please, and confirm that this works the way you want it?

Thanks a lot for your testing, and diving into the code to make these patches.

Updated by Jérôme Schell about 1 year ago

Jonathan Clarke wrote:

Can you test, please, and confirm that this works the way you want it?

I've just tested the new core revision in my ldap2ldap and db2ldap setup. All seems to work fine. userPassword and jpegPhoto attributes are ok. Creation and comparison are handled fine.

Thanks Jonathan.

Updated by Jonathan Clarke about 1 year ago

  • Status changed from Feedback to Closed

Great! Thanks for the feedback. I'm closing this issue as it's resolved now.

Also available in: Atom PDF