DATAREDIS-425 - Add Support for basic CRUD and finder Operations backed by Hashes and Sets.

[christophstrobl]

We now enable storing domain object as a flat Redis HASH and maintain additional SET structures to enable finder operations on simple properties.

  @Keyspace("persons");
  class Person {

    @Id String id;
    @Indexed String firstname;
    String lastname;

    Map<String, String> attributes;

    City city;

    @Reference Person mother;
  }

The above is stored in the HASH with key persons:1 as

  _class = org.example.Person
  id = 1
  firstname = rand
  lastname = al’thor
  attributes.[eye-color] = grey
  attributes.[hair-color] = red
  city.name = emond's field
  city.region = two rivers
  mother = persons:2

Complex types are flattened out to their full property path for each of the values provided. If the properties actual value type does not match the declared one the _class type hint is added to the entry.

  city._class = CityInAndor.class
  city.name = emond's field
  city.region = two rivers
  city.country = andor

Map and Collection like structures are stored with their key/index values as part of the property path.

  list.[0]._class = DomainType.class
  list.[0].property1 = ...

  map.[key-1]._class = DomainType.class
  map.[key-1].property1 = ...

Properties marked with @Reference are stored as semantic references by just storing the key to the referenced object HASH instead of embedding its values.

   mother = persons:2

Please note that referenced objects are not transitively updated/saved and that lazy loading of references will be part of future development.

A save operation therefore executes the following:

  # flatten domain type and add as hash
  HMSET persons:1 id 1 firstname rand …

  # add the newly inserted entry to the list of all entries of that type
  SADD persons 1

  # index the firstname for finder lookup
  SADD persons.firstname:rand 1

Simple finder operation like findByFirstname use SINTER to find matching

  SINTER persons.firstname:rand
  HGETALL persons:1

Besides resolving an index via the @Index annotation we also allow to add custom configuration via the indexConfiguration attribute of @EnableRedisRepositories.

  @Configuration
  @EnableRedisRepositories(indexConfiguration = CustomIndexConfiguration.class)
  class Config { }

  static class CustomIndexConfiguration extends IndexConfiguration {

    @Override
    protected Iterable<RedisIndexDefinition> initialConfiguration() {
      return Arrays.asList(
        new RedisIndexDefinition("persons", "lastname"),
      );
    }
  }

[thomasdarimont]

Cool stuff indeed! Well done Christoph!

While it is definetly cool to have those indexing capabilities based on redis, I wonder whether this approach would really scale - I mean yes you can generate index helper structures in redis but does this really mean one should do this?

I've seen some systems that stored domain objects for fast access by key in redis (hashes) in combination with indexing the domain objects with a search service like elasticsearch or solr.

Lookups by key can then be answered by just querying redis.
In order to retrieve domain objects with more complex queries based on field vales, one simply queries the searchservice which then returns the key where the actual domain object can be looked up with.

The bottom line is: it would be cool to be able to plug-in an query / index services on entity / repository level that could be implemented in a completly different technology, e.g. redis and elasticsearch (in this example).

Cheers,
Thomas

[christophstrobl]

Thanks @thomasdarimont. The idea of having additional structures for data lookups is not to replace a fully fledged search service, but to have a convenient way of finding data by specific attributes. Like having http session data stored in Redis that you do not only want to look up by the session id but also by eg. a users email. Having that said and given the responsiveness of Redis I do think that this approach serves well. But true, the culprit hides in how it's used.

[thomasdarimont]

I totally get that - I was just wondering how far this would go ;-)
The idea I wanted to point out was that it would be cool to have some hooks (without any AOP / AspectJ trickery, perhaps via some kind of EntityListener in the sense of JPA built into the Repository infrastructure) to (amongst other things) easily allow search engines to index newly saved / updated domain objects.

The basic mechanics are already there (JPA EntitListener, MongoDB events or just a RepositoryInterceptor) etc. - wouldn't it be nice to simply call save(..) on a Repository with a domain object which would then also be automatically indexed with elasticsearch or SOLR (after TX completes)?
I could think of something like @EnableEntitySearch / @EnableEntityIndexing that would then use the search service of choice (e.g. elasticsearch or SOLR) based on what's on the classpath.

Currently users have to write additional code to index a saved entity to the search service which could be seen as a bit of boilerplate / cross-cutting concern (searchability?).

Perhaps this would make a nice new feature for SD Commons, don't you think?
Shall I create a JIRA for that?

[christophstrobl]

@thomasdarimont you're already one step ahead. Though I see the use case for that I'm actually a little torn about it. Care to elaborate that at SpringOne? Maybe an issue would help us remember ;)

[mp911de]

Redis Repositories require currently @RedisHash on domain classes to create repositories. RepositoryConfigurationDelegate enables strict repo config mode because it discovers KeyValueRepositoryFactory and RedisRepositoryFactory.