Uniqueness Issue

[mbroecheler]

The uniqueness of MASTER_ID is not ensured in the following example:

https://gist.github.com/anonymous/7579852

[dalaro]

This test has its own tiny project now:

https://github.com/dalaro/titan-add-vertex-test

Observations:

• I reproduced this error every time I ran the test against titan-bdb using default settings
• I never produced the error against titan-cassandra (135 attempts)
• Changing from BDB version 5.0.73 to 5.0.97 had no effect
• Changing from default isolation (repeatable read) to serializable did have an effect, but surprisingly, it did not completely stop the uniqueness violation. It merely reduced the incidence to somewhere around 1 in every 50~100 trials.
• Disabling BDB lock wait, so that attempts to claim unavailable locks fail immediately instead of waiting up to the timeout, seemed to have no obvious effect on the incidence rate of the uniqueness violation, though it did produce more voluminous inter-thread write conflicts
• Despite squinting at the relevant titan-core code, I can't explain how this uniqueness violation can happen under BDB JE serializable transaction isolation. It seems like getVertices(...) in addPropertyInternal should take a reader lock on the relevant vertex index key, and this happpens way before the writes in commit(). There's a good technical argument for using RMW here to reduce spurious deadlock, but I don't see how using RMW would protect against the uniqueness violation. Serializable should already do it.

I'm clearly missing something somewhere...