E11000 duplicate key error collection

[KOSSOKO]

Hi,
First thanks for the wonderfull plugin.

I have two classes Agency and Country with OneToOne relation

@Document
public class Agency {
	
	@Id
	@JsonIgnore
	@Getter
	private String id;
	
	@Getter @Setter
	@Indexed(unique = true)
	private String agencyCode;
	
	//The login of the agency
	@Getter @Setter
	private String login;
	
	//The password of the agency
	@Getter @Setter
	private String password;
	
	@JsonIgnore
	@OneToOne(fetch=FetchType.EAGER)
    @JoinProperty(name = "country")
	@Getter @Setter
    private  Country country;
    
//    @OneToMany(fetch=FetchType.EAGER, cascade = CascadeType.PERSIST)
//    @JoinProperty(name="mobilemoney")
//    private List<MobileMoneyOperator> mobileMoneyService;
    

	/**
	 * constructor with required parameters
	 * @param agencyCode
	 * @param login
	 * @param password
	 * @param country
	 */
	public Agency(String agencyCode, String login, String password, Country country) {
		super();
		this.agencyCode = agencyCode;
		this.login = login;
		this.password = password;
		this.country = country;
	}    
}

public class Country  {

	@Id
	@JsonIgnore
	private String id;
	
	@Indexed(unique = true)
	private	String countryCode;
	
	//The name of the country
	@Indexed(unique = true)
	private String name;
	
	@OneToOne(mappedBy = "country", fetch = FetchType.EAGER)
    private Agency agency;

}

My use case :

• I Created many countries first, then I created each Agency by giving to each agency the country code.

Problem:
When I create a country with an agency to null, the first time it is good. The 2nd time, I got the error

Caused by: com.mongodb.MongoWriteException: E11000 duplicate key error collection: gara-cash-transaction.country index: agency.agencyCode dup key: { agency.agencyCode: null }
	at com.mongodb.client.internal.MongoCollectionImpl.executeSingleWriteRequest(MongoCollectionImpl.java:1055) ~[mongodb-driver-3.11.1.jar:na]
	at com.mongodb.client.internal.MongoCollectionImpl.executeInsertOne(MongoCollectionImpl.java:498) ~[mongodb-driver-3.11.1.jar:na]
	at com.mongodb.client.internal.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:482) ~[mongodb-driver-3.11.1.jar:na]
	at com.mongodb.client.internal.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:476) ~[mongodb-driver-3.11.1.jar:na]

This is probably do to the fact that I added in country

@OneToOne(mappedBy = "country", fetch = FetchType.EAGER)
    private Agency agency;

How is it possible to create all countries first with Agency to null, then to add for each country an agency ?

null is detected as key, in agency.agencyCode, but I don't want him to create cascade.
E11000 duplicate key error collection: gara-cash-transaction.country index: agency.agencyCode dup key: { agency.agencyCode: null }

Thanks in advance.

[kaiso]

Hello, Thank you for using RelMongo
I think your problem is not related to RelMongo, cascading operations is optional and configurable.
You have @indexed annotation on agencyCode so MongoDB does not allow it to be null, so you have to provide a value for this attibute. can you test by providing values for all indexed attributes before saving? thanks.

[santino83]

Hello, I have the same problem with a different annotation: OneToMany

I have two documents, Parent <- OneToMany -> child . The child document has indexes. For design reasons, first I insert the children, then I create the parent, fill up the children field with already-saved documents, and save. The first time I save one Parent, all works, but if I insert a new Parent, Mongo throws the exception "Duplicate Key" (referencing the first unique index in the child document). If I remove the indexes, all works... Seems OneToMany doesn't work when indexes are present, or am I wrong doing something?

[kaiso]

Hello @santino83 can you check where the index is created, is it in the parent or child?

[kaiso]

Hello @santino83 @KOSSOKO , I've reproduced the error you had and presently the @indexed does not work properly when it is used on child associations with relmongo, in fact spring is considering the documents as embedded so it will create indexes in parent collections.

A workaround to this is to create the indexes through mongoOperations (mongoTemplate) on the child class directly instead of using @indexed annotation like this :

mongoOps.indexOps(User.class).ensureIndex(new Index().on("name", Direction.ASC));

I will keep this issue opened to try to find a solution to make @indexed work on child associations.

Thank you and sorry for this.

[KOSSOKO]

Thanks for your answer. Yes, it's a good workaround to create the index dynamically. I will try it and give a feedback.

[santino83]

Thank you, I'll try the workaround too. Infact, the error was prettly clear, mongoTemplate tries to manipulate indexes, beacuse knows nothing about your annotation. I think it needs a lot of investigation to find out a robust solution to avoid this behavior.

Thanks for your job

[santino83]

I can confirm you that removing indexed annotations on child document fix the error. Obviously, I had to create indexes programmatically using mongoTemplate IndexOperations. I'll look forward to read news about a fix for this "bug". Thank you

[kaiso]

hi @santino83 @KOSSOKO , I released a candidate version 3.1.0-RC1 which include a fix, can you test it with @indexed annotation and give your feedback please. I will release the 3.1.0 version if your tests are concluding.

[santino83]

I cannot reproduce the error, as I cannot use v3.1.0 in my main project (as it uses spring boot < 2). I've created a new project with just spring boot/spring starter mongo, embeeded mongodb and 2 documents, Parent and Child with OneToMany. I had different results, sometimes 3.0.1 throws error, sometimes no error. I'm only sure that 3.1.0-RC1 never throws errors (I tested with spring boot >= 2.0 < 2.2). Maybe I have some problem with my maven. Here is my code:

Parent document:

@Document(collection = "Parent")
@Accessors(chain = true)
public class ParentDocument
{
    
    @Id
    @Getter
    private ObjectId id;
    
    @Getter @Setter
    private String name;
    
    @OneToMany(cascade = CascadeType.NONE, fetch = FetchType.EAGER)
    @JoinProperty(name = "children")
    @Getter @Setter
    private List<ChildDocument> children = new ArrayList<>();
    
}

Child document:

@Document(collection = "Child")
@Accessors(chain = true)
public class ChildDocument
{
    
    @Id
    @Getter
    private ObjectId id;
    
    @Getter @Setter
    @Indexed(unique = true)
    private String name;
    
}

App configuration:

@Configuration
@EnableRelMongo
@EnableMongoRepositories
public class AppConfig
{
    
}

Test:

@DataMongoTest
@RunWith(SpringRunner.class)
public class ParentRepositoryTest
{
    
    @Autowired
    ParentRepository parentRepo;
    
    @Autowired
    ChildRepository childRepo;
    
    @Before
    public void setup()
    {
        parentRepo.deleteAll();
        childRepo.deleteAll();
        
        for(int i = 0; i< 2; i++){
            ChildDocument child = new ChildDocument();
            child.setName("name_"+i);
            childRepo.save(child);
        }
        
        assertTrue(childRepo.findAll().size() == 2);
    }
    
    @Test
    public void test_save_children()
    {
        ChildDocument child1 = new ChildDocument();
        ChildDocument child2 = new ChildDocument();
        
        child1.setName("EQUAL_NAME");
        child2.setName("EQUAL_NAME");
        
        try{
            childRepo.save(child1);
            childRepo.save(child2);
            fail("Should throw exception");
        }catch(Exception ex){
            assertTrue(true);
        }
    }
    
    @Test
    public void test_save_parent_children()
    {
        ParentDocument parent = new ParentDocument();
        parent.setName("parent_1");
        parent.setChildren(childRepo.findAll()); //load pre-existent children
        
        ParentDocument saved = parentRepo.save(parent);
        assertTrue("parent_1".equals(saved.getName()));
        assertTrue(saved.getChildren().size() == 2);
        
        ParentDocument loaded = parentRepo.findById(saved.getId()).get();
        assertTrue("parent_1".equals(loaded.getName()));
        assertTrue(loaded.getChildren().size() == 2);
        
    }
    
    @TestConfiguration
    @EnableRelMongo
    public static class Config{
        
    }
}

where ParentRepository and ChildRepository are simple mongoRepository interface.

Pom (last i've used):

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http:https://maven.apache.org/POM/4.0.0" xmlns:xsi="http:https://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http:https://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.9.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>it.santino83</groupId>
    <artifactId>demorelmongo</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>demorelmongo</name>
    <description>Demo project for Spring Boot</description>

    <properties>
        <java.version>1.8</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-mongodb</artifactId>
        </dependency>
        
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
        </dependency>
        
        <!-- mongo extra features -->
        <dependency>
            <groupId>io.github.kaiso.relmongo</groupId>
            <artifactId>relmongo</artifactId>
            <version>3.0.1</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>org.junit.vintage</groupId>
                    <artifactId>junit-vintage-engine</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>de.flapdoodle.embed</groupId>
            <artifactId>de.flapdoodle.embed.mongo</artifactId>
            <scope>test</scope>
        </dependency>
        
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

and it works with relmongo 3.0.1 and 3.1.0-RC1...

log using 3.0.1:

2019-11-19 23:11:02.402 DEBUG 18758 --- [           main] org.mongodb.driver.protocol.command      : Sending command '{ "insert" : "Parent", "ordered" : true, "documents" : [{ "_id" : { "$oid" : "5dd4687688e8be494636712a" }, "name" : "parent_1", "_class" : "it.santino83.demorelmongo.documents.ParentDocument", "children" : [{ "_id" : { "$oid" : "5dd4687688e8be4946367128" }, "_relmongo_target" : "Child" }, { "_id" : { "$oid" : "5dd4687688e8be4946367129" }, "_relmongo_target" : "Child" }] }] }' with request id 16 to database test on connection [connectionId{localValue:2, serverValue:2}] to server localhost:37999

log using 3.1.0-RC1:

2019-11-19 23:14:17.369 DEBUG 27481 --- [           main] org.mongodb.driver.protocol.command      : Sending command '{ "insert" : "Parent", "ordered" : true, "documents" : [{ "_id" : { "$oid" : "5dd4693988e8be6b59e7c31d" }, "name" : "parent_1", "_class" : "it.santino83.demorelmongo.documents.ParentDocument", "children" : [{ "_id" : { "$oid" : "5dd4693988e8be6b59e7c31b" }, "_relmongo_target" : "Child" }, { "_id" : { "$oid" : "5dd4693988e8be6b59e7c31c" }, "_relmongo_target" : "Child" }] }] }' with request id 15 to database test on connection [connectionId{localValue:2, serverValue:2}] to server localhost:33951

I'll try again tomorrow

[kaiso]

Hello @santino83 ,

• first you have to use the version 3.1.0-RC1 which contains the fix and not the 3.0.1.
• I recommand the use of the last spring-boot version 2.2.1.RELEASE which contains the same version of spring-data-mongodb.

it's good news that it does not throw errors. I will release the final version soon.

[KOSSOKO]

Hum, ok, so I need also to upgrade my Spring version from 2.2.0.RELEASE to 2.2.1.REALEASE to test. Ok, I will give test and give you a feedback

[kaiso]

The Final release RelMongo 3.1.0 has been published.