Project: Implement Spring Data to LibreHealth Toolkit

@sunbiz I’ve made a very primitive filtering structure, here’s a blog post about it.
Although when trying to implement pattern based matching or partial search, I get indexing errors i.e LIKE cannot be used on non-indexed columns. I’m looking deeper into indexing in cassandra. Any ideas?

Update: Here’s the repo link.

The indexing errors were solved by creating a custom SASI Index for certain columns. I made an annotation to do this, here’s how a sample Patient looks like now.
@sunbiz Could you please elaborate on your solution for annotating the HAPI FHIR structures issue,

@sunbiz I have finally figured out the null pointer errors, Cassandra was unable to map java.lang.Object of the UserData property of HAPI FHIR structures. I solved it by annotating the getter of userdata with @CassandraType(type = DataType.Name.MAP, typeArguments = {DataType.Name.VARCHAR, DataType.Name.VARCHAR}) in spring 2.1.0 SNAPSHOT (I’ll try this in the RELEASE version today).
This solved the mapping errors as well since marking the getters with @Id annotations and subclasses with @Table got the models working with Cassandra.
I’ll push this today after configuring all the converters and indexed fields.

@sunbiz I’ve pushed the changes after configuring all the converters for the Patient model and disabled saving unwanted properties in the database as well. This is how a Patient model looks like now,

CREATE TABLE cassandra.patient (
    patient_id text PRIMARY KEY,
    active boolean,
    address text,
    animal text,
    birthdate timestamp,
    communication text,
    contact text,
    contained text,
    deceased text,
    empty boolean,
    extension text,
    formatcommentspost text,
    formatcommentspre text,
    gender text,
    generalpractitioner text,
    id text,
    identifier text,
    implicitrules text,
    language text,
    link text,
    managingorganization text,
    managingorganizationtarget text,
    maritalstatus text,
    meta text,
    modifierextension text,
    multiplebirth text,
    name text,
    photo text,
    primitive boolean,
    resource boolean,
    telecom text,
    text text,
    userdata map<text, text>
) 

So you are extending the HAPI DSTU3 structures, but since you are extending and annotation, it is not serving the purpose. The purpose of using the HAPI structures was that they usually keep in sync with the standard, and if you have to rewrite each property of their class, it basically doesn’t make much sense.

I havent been able to figure out a way to annotate without extending either. I like your approach and it is fine for the moment, but we still should continue to find other way, instead of replicating all their model code.

2 Likes

@sunbiz I’ve added a data loader similar to the pre-requisite task project which retrieves sample resources from a FHIR server and the data is saved in the database just fine (except for StackOverFlow errors with Jackson which were solved using Gson instead), but retrieving data back from the database gets stuck for some reason without throwing any errors whatsoever.
I’ve succesfully tried logging the queries being executed so the error is not on the cassandra part. It seems the error must be somewhere in data conversion, I’ll keep you posted on this.

PS: Data retrieval worked fine when I was using a sample Patient model.

1 Like

@yashdsaraf by when do you think you will be able to support the CRUD on the full Patient resource at least? I want to start thinking about the way to distribute this such that it can be an EHR platform. i.e. support Authentication, module assembly etc?

@sunbiz As I said in my previous post I’m able to save the Patient which means implementing the Update and Delete functionalities should be quite straight forward as well. Once the READ operation is entirely sorted out, others shouldn’t take much longer. I’m working on fixing the no output issue I mentioned earlier, once that’s done I’ll be able to give you a much better answer as to when the CRUD support will be complete.

That sounds exciting! :grinning:

1 Like

@sunbiz I’ve tried debugging the app to find where the app gets hung up, all I know this far is that CQL queries are executed successfully and the ResourceConverter is the last one to get executed before the app gets stuck. I also tried to make a custom model using just a single data type from HAPI FHIR structures e.g IdType where the app didn’t get stuck but Jackson threw some mapping errors which I’m afraid will come later in our app as well. I’m currently trying to get a non reactive version of the app working, I’ll get back to you with the progress in some time.

1 Like

@sunbiz The non reactive version didn’t work either. It seems I’m back to square one regarding this error, it would be great if you could give the code a try https://gitlab.com/yashdsaraf/reactive-lh-toolkit.

Here’s what I know so far,

  • Saving works but the app freezes on select operation without any error or output
  • Select queries are successfully executed (you can enable logging from application.properties) so the error is not in the CQL queries
  • I used the debugger to track all the converters and they were successfully executed as well
  • Execution doesn’t reach the stage where the CustomPatient object is converted to JSON to be returned to the browser. (I know this because Jackson errors out on UriType fields and there are no such errors in this case.)

Any ideas on how to solve or debug this?

@sunbiz @namratanehete I’m working on the CREATE, UPDATE and DELETE operations for Patient.
I’m using Http - FHIR v5.0.0 for reference.
It says that

servers MAY choose to allow clients to PUT a resource to a location that does not yet exist on the server - effectively, allowing the client to define the id of the resource.

Do we want to allow creating a resource with unknown id using PUT?

Yes, I think we should allow it.

1 Like

@sunbiz I’ve implemented CRUD on sample patient model, referring to https://www.hl7.org/fhir/http.html. Here’s the gist of the code.
I also figured out a way out of Jackson mapping errors, since we can’t directly annotate the HAPI models, we can use Jackson Mixins to overcome stackoverflow errors and we can choose which properties are displayed and which ones aren’t.
For the READ error, I created a sample spring project with spring boot snapshot version and it is throwing some mapping errors (unlike our original app which just keeps on loading without any output or error whatsoever). I’d like to go down this rabbit hole for now and see where it leads. I’ll report back the progress in some time.

1 Like

@sunbiz The snapshot eventually started freezing on READ as well. So I took the long way around and started eliminating all the properties in Patient to isolate the cause of the error. Finally after trimming down Patient to just a couple of properties the READ operation started working. I will start enabling the properties that need to be persisted today, I’m confident that this will finally complete the READ operation.

1 Like

@yashdsaraf do you have a branch where I can test this on? I would like to see @prashadi and you working off the same codebase now, that you are both fairly well into your projects.

Can you please provide your repos that I can use to cherry-pick and move to the librehealth org in gitlab?

@sunbiz I found the error as to why the app got stuck on READ operation. Apparently the spring cassandra’s mapping converter could not find setters for some properties. For e.g For Animal property I was getting

java.lang.NoSuchMethodError: org.hl7.fhir.dstu3.model.Patient.setAnimal(Lorg/hl7/fhir/dstu3/model/Patient$AnimalComponent;)V 

But using a very strange workaround of marking animal as transient and creating getter/setter with a different name got the app working.

@Transient
  @Override
  public AnimalComponent getAnimal() {
    return super.getAnimal();
  }

  @Column("animal")
  public AnimalComponent getDummyAnimal() {
    return super.getAnimal();
  }

  public void setDummyAnimal(AnimalComponent value) {
    super.setAnimal(value);
  }

My guess is this error was not recognized by the reactive stream and it just kept waiting for the data. This is why the app froze every time a GET query was fired.
I’ve pushed the code with updated mixins, converters and this workaround at link.
I’d recommend to not cherry pick the commits just yet, since we know the workaround for the error now, I can get the whole patient model working. I’ll try to push the whole working patient model by tonight as well as a blog post about the workaround.

ok, I’ll wait, but please update us soon. We are already a week into phase 2 of GSoC

@sunbiz I’ve pushed the code with working READ operation and modeled Patient to look like https://www.hl7.org/fhir/patient.html.
There are some issues where the data retrieved is showed as JSON instead of primitive types, I’m working on fixing that now.

1 Like

@sunbiz Here’s the blog post regarding the workaround and CRUD operations for sample Patient model - https://medium.com/@yashdsaraf/gsoc-librehealth-work-around-for-app-freezing-on-read-605dd2ed5c25

@sunbiz I’ve added the save and update functions for Patient and fixed the incorrect serialization issues for a lot of properties. There are still some issues when generic types such as Type and Resource are serialized, I’m working on fixing those.