the gorma Plugin

gorma is a goa plugin that makes it possible to describe database models. The gorma code generator uses the model definitions to generate code that automatically creates media types from models. More documentation to come.

Overview

goa is a DSL for the end-user experience of your API. It describes the inputs and outputs in exact detail. gorma extends this experience to the data storage layer. gorma adds a new DSL to describe your storage layer and the relationships between your models. Your gorma specification lives in the same design folder as your goa DSL. Here’s an example:

package design

import (
	"github.com/goadesign/gorma"
	. "github.com/goadesign/gorma/dsl"
)

var _ = StorageGroup("CongoStorageGroup", func() {
	Description("This is the global storage group")
	Store("postgres", gorma.Postgres, func() {
		Description("This is the Postgres relational store")
		Model("User", func() {
			BuildsFrom(func() {
				Payload("user", "create")
				Payload("user", "update")
			})
			RendersTo(User)
			Description("User Model Description")
			HasMany("Reviews", "Review")
			HasMany("Proposals", "Proposal")
			Field("id", gorma.Integer, func() {
				PrimaryKey()
				Description("This is the User Model PK field")
			})
			Field("created_at", gorma.Timestamp, func() {})
			Field("updated_at", gorma.Timestamp, func() {})
			Field("deleted_at", gorma.NullableTimestamp, func() {})
		})

		Model("Proposal", func() {
			BuildsFrom(func() {
				Payload("proposal", "create")
				Payload("proposal", "update")
			})
			RendersTo(Proposal)
			Description("Proposal Model")
			BelongsTo("User")
			HasMany("Reviews", "Review")
			Field("id", gorma.Integer, func() {
				PrimaryKey()
				Description("This is the Payload Model PK field")
			})
			Field("title", func() {
				Alias("proposal_title")
			})
			Field("created_at", gorma.Timestamp, func() {})
			Field("updated_at", gorma.Timestamp, func() {})
			Field("deleted_at", gorma.NullableTimestamp, func() {})
		})

		Model("Review", func() {
			BuildsFrom(func() {
				Payload("review", "create")
				Payload("review", "update")
			})
			RendersTo(Review)
			Description("Review Model")
			BelongsTo("User")
			BelongsTo("Proposal")
			Field("id", gorma.Integer, func() {
				PrimaryKey()
				Description("This is the Review Model PK field")
			})
			Field("created_at", gorma.Timestamp, func() {})
			Field("updated_at", gorma.Timestamp, func() {})
			Field("deleted_at", gorma.NullableTimestamp, func() {})
		})

		Model("Test", func() {
			Description("TestModel")
			NoAutomaticIDFields()
			Field("created_at", gorma.Timestamp, func() {})
			Field("updated_at", gorma.Timestamp, func() {})
			Field("deleted_at", gorma.NullableTimestamp, func() {})
		})
	})
})

Like goa, gorma’s DSL is really a set of Go functions that builds a configuration behind the scenes. gorma uses this configuration to generate a database access package in the models directory of your project. Let’s walk through the DSL in the above example.

Storage Group

The StorageGroup specification is the root of your gorma configuration. On its own it creates nothing, but it serves as the container for all of the storage specifications for your project. There may be only one StorageGroup defined in your project. As with nearly all the DSL in goa and gorma, you can add a Description("...") specification to your StorageGroup.

Store

A Store represents a single data store; today that means a relational database. This restriction may change in the future based on contributions, demand and need. The Store has a name and a type:

	Store("postgres", gorma.Postgres, ...)

Today these identifiers aren’t used in code generation, but may be used in the future to change the generated code in database specific ways. The name isn’t special–you can use any name you like.

The example Store above has a Description and several Model definitions:

	Store("postgres", gorma.Postgres, func() {
		Description("This is the Postgres relational store")
		Model("User", func() { ...

Your Store may contain an arbitrary number of models, and the models you specify aren’t limited to the ones that are specified in your goa DSL.

Model

A Model represents a single struct or domain object that maps to a database table. The Model DSL is the most flexible DSL in gorma, and it’s where you’ll do most of your specification. Models generated by gorma use gorm to access your database. This means that gorma supports any database driver that gorm supports. All testing in gorma is currently done in PostgreSQL. gorma uses no database specific functionality, so it should work equally well across supported databases.

Let’s break down the Model definition and see what’s happening:

		Model("User", func() {
			BuildsFrom(func() {
				Payload("user", "create")
				Payload("user", "update")
			})
			RendersTo(User)
			Description("User Model Description")
			HasMany("Reviews", "Review")
			HasMany("Proposals", "Proposal")
			Field("id", gorma.Integer, func() {
				PrimaryKey()
				Description("This is the User Model PK field")
			})
			Field("created_at", gorma.Timestamp, func() {})
			Field("updated_at", gorma.Timestamp, func() {})
			Field("deleted_at", gorma.NullableTimestamp, func() {})
		})

The first stanza in the configuration is the BuildsFrom DSL. This BuildsFrom tells gorma that this model will accept data that is input from the payloads that are sent to the “create” and “update” actions of the User resource. This is important because gorma will automatically generate conversion functions that convert goa’s strongly typed Payload objects into a User model suitable for the Create and Update methods in the data access code.

The next line specifies that the User model RendersTo the “User” media type in your goa design. Again, gorma will automatically generate conversion functions for you that convert the database model into strongly typed goa Views. Because of the power of the goa DSL, gorma is able to generate code to create the full view object graph. If your User media type specifies that it will return goa Links, gorma will generate the code to create and embed those links in your view. Similarly, if your view specifies that it returns nested models, gorma will retrieve and populate those models too. Here’s an example of a goa Media Type that specifies embedded objects in it’s definition:

var Proposal = MediaType("application/vnd.proposal+json", func() {
	Description("A response to a CFP")
	Reference(ProposalPayload)
	Attributes(func() {
		Attribute("id", Integer, "ID of user")
		Attribute("href", String, "API href of user")
		Attribute("title", String, "Response title")
		Attribute("abstract", String, "Response abstract")
		Attribute("detail", String, "Response detail")
		Attribute("reviews", CollectionOf(Review), "Reviews")
	})
	Links(func(){
		Link("reviews")
	})
	View("default", func() {
		Attribute("id")
		Attribute("href")
		Attribute("title")
		Attribute("abstract")
		Attribute("detail")
		Attribute("reviews")
		Attribute("links")
	})
	View("link", func() {
		Attribute("id")
		Attribute("href")
		Attribute("title")
	})
})

Of particular note here is the Attribute("reviews", CollectionOf(Review), "Reviews") line. The goa DSL specifies that this Proposal media type has an attribute called “reviews” that is actually a collection of the Review media type. This attribute is included in the “default” view of the Proposal media type. In order for this to work in gorma you need to define the relationship between the models in your DSL. Going back to the User model DSL you can see how gorma does this:

		Model("User", func() {
			BuildsFrom(func() {
				Payload("user", "create")
				Payload("user", "update")
			})
			RendersTo(User)
			Description("User Model Description")
			HasMany("Reviews", "Review")
			HasMany("Proposals", "Proposal")
			...

We’ve adopted the Active Record nomenclature for model relationships. gorma DSL supports HasMany, HasOne, BelongsTo, and ManyToMany relationships. The DSL specifies the relationships between gorma models and needs to match the relationships you specify in your goa Media Types. As in the Media Type definition above, where Proposal includes a collection of Review, the Proposal model in gorma must declare a HasMany relationship with the Review model.

In this Model definition example, we’ve included the created_at, updated_at, and deleted_at fields, but it’s not necessary to specify them. gorma will automatically include them unless you specify the NoAutomaticTimestamps() DSL in your Store or Model definition. Similarly, gorma will create a primary key field called id of type Integer (however that maps to your database’s storage model). You can suppress this with the NoAutomaticIDFields() DSL in the Store or Model definition.

Fields

gorma will attempt to populate the fields of your model based on the fields in your BuildsFrom DSL. This means that if the payloads you reference contain a set of fields, your gorma model will get those fields too. Sometimes you need to specify extra information about those fields, like when you want to add a secondary index. Other times, you want fields in your Model that aren’t included in any of your Payloads. The Field DSL allows you to specify field definitions for your Models.

Here’s an excerpt from a Proposal model definition that specifies several properties on the Proposal Model’s fields:

		Model("Proposal", func() {
			...
			Field("id", gorma.Integer, func() {
				PrimaryKey()
				Description("This is the Payload Model PK field")
			})
			Field("title", func() {
				Alias("proposal_title")
			})
			Field("created_at", gorma.Timestamp, func() {})
			Field("updated_at", gorma.Timestamp, func() {})
			Field("deleted_at", gorma.NullableTimestamp, func() {})
			...
		})

The “id” field is explicitly set here, but as mentioned above it doesn’t need to be. It also includes the PrimaryKey() definition in the field’s DSL. If you have compound primary keys, you can specify them by creating multiple field definitions in your Model with the PrimaryKey() tag.

We’ve also specified an Alias for the “title” field, which tells gorma that the title field in the model will be stored in the proposal_title column in the database. You can specify an Alias for both Model and Field definitions. If you specify an Alias on a table, you’re changing the name of the table in the database.

Running gorma

Generate your models by using the gen feature of the goagen command:

	goagen --design=gopath/to/your/project/design/ gen --pkg-path=github.com/goadesign/gorma

Expectations

gorma makes certain expectations, and you shouldn’t deviate too far from them if you want all the automatic wiring to work well.

  • Your Model relationships need to match the implicit relationships you specify in your Media Types. When you specify that a Media Type has an Attribute that is actually another Media Type you’re specifying a relationship. That relationship must be also reflected in your gorma DSL.
  • Naming: Name your Models with CamelCase. Name your Fields with snake_case. gorma always tries to do the right things with naming and conversions, but keeping this standard will help with edge cases that aren’t yet tested.
  • gorma generates your models in a folder called models. You can override this by appending -- --pkg=otherfolder when you call goagen.

Limitations

  • Gorma expects that all of your keys (primary and foreign) will be int types. In the future we may support non-integer primary keys.
  • Gorma is currently tested extensively against PostgreSQL. It should work well against other databases supported by Gorm, but if it doesn’t please file a bug.
  • The methods generated by gorma are basic case methods for data manipulation by primary key. It’s possible to add lookup functions in the future by other fields in the model. Pull requests gratefully accepted.
  • Gorm ~ and any ORM, really ~ won’t generate optimal SQL for every use case. If you need to optimize a particular case, see the note under “Other Notes” below about adding additional methods to your Models, and hand-write your SQL. gorma exposes the underlying database handle so you can drop down to Go’s db/sql package and write anything you want in your custom methods.

Other Notes

  • gorma generates your models into the models folder of your project’s root. It will overwrite these models every time you use goagen, but it will only overwrite files that it creates. You can add extra files to this directory to specify additional functionality for your models as long as you use a file name that won’t conflict with gorma’s naming convention: model.go and model_helper.go. We recommend something like model_addons.go to keep your extra code from being overwritten.
  • Gorm’s update model is a little bit scary. If you are writing custom functions, be cautious about how you send the updates to Gorm. gorma accounts for this by using pointers on any field that can be null, and checking each field before adding the field to the update clause in the Update functions that it generates. When you’re writing custom Update methods, don’t send a goa payload directly to the Save() method in Gorm, or you will overwrite columns that are populated in the database with the fields that aren’t populated in your model. Instead, call the Update method in Gorm and explicitly pass in the fields and values you want to update.
Extending goa with Plugin DSLs