Developing group video themes

We provide tools to allow designers to create custom group video themes. Fundamentally, a group video theme consists of an interface that collects media from a group of people using a set of input elements with customizable prompts and constraints, and it contains logic to stitch the corresponding input media together into a themed video. These themes are complex — basic software development / JavaScript knowldege is required. See also our documentation for our web API.

Getting started

• Start developing a new group video theme from the design portal (registration required).
• Click on the "Create new" button.
• On the page, enter a user-facing name and description for your theme.
• Click on the button to edit the theme code. You'll probably want to use your own editor and simply paste you code there when you're redy to test. Make some updates and click "Save".
• Click on the button to see how your theme looks to a project organizer.
• Drag media from our library of examples in the left column to test your theme. This is extra important, because we'll use the example you build as the user-facing preview of your theme on the group video page.
• Use the button to add / manage any static assets you want to incude directly in your theme.
• Use the and buttons to see the actual data that is passed to and from your theme's build function to help you debug.
• When you're satisfied, submit your theme for our review by clicking the button.

Theme modules

• get sections() returns a list of the user-facing input sections based on the current state of this.project
• async build() returns an object in the form of an ellacard-video, which we can play back live in a web browser and render to a video file

• assets a function that allows you to use static assets you've uploaded via the page
• utils contains various helper classes and functions

const utils = require('utils');

class Builder extends utils.AbstractProjectBuilder {

  get sections() {
    return [ <input section definitions here> ];
  }

  build() {
    <logic here>
    return this.buildResources();
  }
}

module.exports = Builder;

Project

{
  created_at: ISO datetime str,
  user_id: UUID str,
  sections: {
    // These sections contain the user input and correspond to the section 'id' keys you defined in your module's 'sections'
    'section-id-1': {
      inputs: [
        {
          created_at: ISO datetime str,
          updated_at: ISO datetime str,
          input_id: UUID str, // A unique ID for this input
          user_id: UUID str, // The unique user ID associated with this input
          version_id: int, // Successive updates to this input will increment the version number
          type: str, // The input type (note that differs from media types)
          data: { ... } // The 'data' depends on the type of this input
        },
        ...
      ]
    },
    ...
  }
}

Input sections

This section lists the input types available to use when you set up the input sections in your theme via the get sections() method. The input spec tab shows the options you have to configure the user input. The project input data tab shows the data property from the output of this.getInputs(…) for the corresponding input type. The builder resources tab shows the output of this.getResources(…) for the corresponding input type — this is generally how you should interact with inputs as they are already packaged as media resources for you (sans automations).

{
  // The section ID must be unique
  id: str,

  // Whether or not we should show this section to only the project organizer
  owner_only: bool,

  // Whether or not this input is required.
  required: false,

  title: str, // Optional user-facing text, each input type has a default title (like "Add a picture" for IMG)
  subtitle: str, // Optional user-facing text, provides more context than just the title itself

  // The user-facing description, or prompt, that tells a user what to do for this section
  details: str,

  type: str, // Must be a valid input type

  // The input specification whose content will vary based on the type
  spec: ...
}

AUDIO_CHOICE

{
  options: [
    {
      name: str,
      src: null | assets('audio_file.mp3'),

      // Optional, only one option should be set as default
      default: true
    },
    ...
  ]
}

IMG

{
  fixed_aspect_ratio: float, // Optional, the ratio of width / height
  preferred_aspect_ratio: float // Optional, defaults to 4x3
}

IMG_GROUP

{
  min: int, // Optional, note that you will never get passed data for an IMG_GROUP input with zero images
  max: int,
  fixed_aspect_ratio: float, // Optional, the ratio of width / height
  preferred_aspect_ratio: float // Optional, defaults to 4x3
}

MULTIPLE_CHOICE

{
  // Optional, if false-ish, the user can only select one option
  // If set to true, the user can select any number of options
  // If an object, the user must select a number of options between the 'min' and 'max' (inclusive). Note that only one of 'min' and 'max' needs to be provided, though both will be respected
  multiple: null | true | { min: int, max: int },

  // Optional, the number of options the user is allowed to write-in (true == 1), defaults to 0
  custom: int,

  // The pre-populated options in the list
  options: [
    {
      text: str,
      default: true // Optional, set to true to have this option be initially checked
    },
    ...
  ]
}

TEXT

{
  // The size of the output image to which the text will be rendered
  size: { w: int, h: int },

  // Text can automatically resize to its container. It will start out at the 'font_size' specified, and as the user enters more text that would overflow the container, the font size will automatically shrink as far as the 'min_font_size' value allows it
  font_size: int,
  min_font_size: int,

  // These default values are always required
  text: str, // Placeholder text (or example text if 'text' is not a configurable option)
  font: 'Nunito', // Font family (must be supported - see below for a list of supported fonts)
  color: '#FEFEFE', // CSS color string
  bold: bool,
  italic: bool,
  underline: bool,

  // This property determines how much a user can configure. If this object is empty, the user won't be able to change anything other than the text itself.
  options: {
    // A user will only be able to edit the default text if you set this property to true
    text: bool,

    // Set to true to show all supported font choices (see list below), or you can explicitly list the fonts you want to let the user choose from - make sure the default 'font' you specified earlier is in this list
    font: bool | array,

    // Set to true to show the default colors as used in the cards (see list below), or you can explicitly list the colors to let the user chosoe from - make sure your default 'color' is in this list
    color: bool | array,

    // Set to true to allow the user to configure these options
    bold: bool,
    italic: bool,
    underline: bool
  }
}

VIDEO

{
  fixed_aspect_ratio: float, // Optional, the ratio of width / height
  preferred_aspect_ratio: float, // Optional, defaults to 16x9
  duration: {
    min_s: int, // Optional (defaults to 0)
    max_s: int // This will also govern the max data size of the input
  },

  // Set to true to present an interface that mutes the uploaded video
  exclude_audio: bool // Optional (defualts to false)
}

Additional input types

Do you need an additional input type to really make your theme work? Please let us know and explain your use case! We're developing more all the time. Note that theoretical theme inputs don't necessarily have to be related to media. For example, a user could be prompted to answer a question or vote on something, and your theme could use that information to help construct the video.

Ellacard-video

{
  // Specify your coordinate system for laying out content. We may render the actual video itself at different scales.
  // Warning: some of the helper functions assume that the size will be set to 1920x1080.
  w: 1920,
  h: 1080,
  media: [
    {
      // A unique ID for this media element across the whole JSON structure
      id: str,

      // The type of this media component
      type: AUDIO | AUDIO_VIDEO | IMG | TEXT | VIDEO,

      // The URL for the native (highest quality) source of this media. Only relevant if not TEXT.
      [src]: str,

      // The following properties apply only to TEXT media elements. When the resource is initially loaded, we use these properties to render an image. From that point on, TEXT is treated like an IMG.
      [size]: { w: int, h: int },
      [font_size]: int,
      [min_font_size]: int,
      [text]: str,
      [font]: Font family str,
      [color]: CSS color str,
      [bold]: bool,
      [italic]: bool,
      [underline]: bool,

      // The offset from the start of the media to begin automations (optional), does not apply to images or text
      t_offset: null | float,

      // Duration in seconds to transition opacity (for visual media) or volume (for audio) at the start / end of the automation (optional)
      fade_in: null | float,
      fade_out: null | float,

      // The intrinsic volume adjustment of an AUDIO or AUDIO_VIDEO element. It will be multiplied by the volume derived from the automations. This value should be in the range [ 0.05, 2 ].
      v: null | float,

      // For AUDIO_VIDEO, VIDEO, and IMG, these properties describe the intrinsic rotation and percentage to crop off from each edge - crop is applied to the native video size, not the size laid out in the automation (optional)
      r: null | 0 | 90 | 180 | 270,
      crop: null | { top: float, left: float, right: float, bottom: float },
      letterbox: null | float,

      // All the properties in the automations will be interpolated between timestamps
      automation: [
        {
          // The absolute timestamp (in seconds) for when this media element begins playback
          t: float,

          // The center coordinates of the media element (for visual media elements only: AUDIO_VIDEO | VIDEO | IMG)
          x: float,
          y: float,

          // The unrotated size of the media element (for visual media elements only)
          w: float,
          h: float,

          // The rotation of the media element (in degrees)
          r: float,

          // For media elements with audio: AUDIO_VIDEO | AUDIO, a value in the range [0, 1] that specifies a percentage of baseline volume - fade in / out will be applied over the top of this value
          v: float
        },

        // Must have at least two automation entries in order of increasing 't'
        ...
      ]
    },
    ...
  ]
}

Helper modules

Assets module

function assets(assetId)

Utils module

function utils.buildFrame(resource, thickness, color, [size])

function utils.buildSlideshowKenBurns(resource, t0, [options])

function utils.layout.getScale(w, h, fitW, fitH, mode)

function utils.layout.scaleForArea(size, area)

function utils.spreadHorizontally(resources, t0, [options])

class utils.AbstractProjectBuilder

utils.AbstractProjectBuilder.getInputs(sectionId=null)

utils.AbstractProjectBuilder.getInput(sectionId)

utils.AbstractProjectBuilder.getResources(sectionId=null)

utils.AbstractProjectBuilder.getResource(sectionId)

utils.AbstractProjectBuilder.addResource(resource)

utils.AbstractProjectBuilder.buildResources()

utils.AbstractProjectBuilder.randomInt(min, max)

Annotated example

const assets = require('assets');
const { AbstractProjectBuilder } = require('utils');

// For this theme, the sections are not dynamic - they do not change based on project input, so we can declare them statically here.
const SECTIONS = [
  {
    id: 'title',
    owner_only: true,
    required: false,
    subtitle: 'Title',
    details: 'Enter a title for your video',
    type: 'TEXT',
    spec: {
      size: { w: 1840, h: 460 },
      font_size: 164,
      min_font_size: 96,
      text: 'Name',
      font: 'Luckiest Guy',
      color: '#FFF',
      bold: true,
      italic: false,
      underline: false,
      options: {
        // We only let the user configure these options
        text: true,
        font: true
      }
    }
  },
  {
    id: 'say-hello',
    owner_only: false,
    required: false,
    type: 'VIDEO',
    subtitle: 'Say hello',
    details: `Add a video of you waving and saying hello!`,
    spec: {
      duration: {
        max_s: 10
      }
    }
  }
];

class Builder extends AbstractProjectBuilder {

  get sections() {
    // Note that the 'project' could be null at this point
    return SECTIONS;
  }

  build() {
    let t = 0;

    // We declared an input section called 'title' that was for the organizer only. When a section has 0 or 1 resources, you can access it using 'getResource' so you can set automations for it
    const titleResource = this.getResource('title');
    titleResource.automation = [
      // The automations are the key to the ellacard video theme. Every media resource in the output file must have at least two automations, though you can set as many as you want. Video renderers will linerly interpolate all the properties between the timestamps in your automations, including volume. For visual media (IMG, TEXT, and VIDEO), you must specify the 'x' and 'y' coordinates of the center of the object, its size as 'w' and 'h', and its rotation 'r' in degrees. For VIDEO and AUDIO, you must specify the 'v' property for volume in the range [0,1].
      { t: 0, x: 960, y: 540, w: 1920, h: 1080, r: 0 },
      { t: 5, x: 960, y: 540, w: 1920, h: 1080, r: 0 }
    ]

    // The section called 'say-hello' open to all contributors. When a section can have more than 1 resource, use 'getResources'
    t += 5;
    for (const resource of this.getResources('say-hello')) {
      resource.automation = [
        { t, x: 960, y: 540, w: 1920, h: 1080, r: 0, v: 1 },
        { t: t + resource.meta.duration, x: 960, y: 540, w: 1920, h: 1080, r: 0, v: 1 }
      ];
      t += 5;
    }

    // You can also add your own resources directly to the builder, like if you had a static background asset to include that you wanted to show throughout the entire video. Note that we provide a means to include assets using our CDN via the 'assets' tab in the design video theme tool. Do not use external URLs.
    background = {
      id: 'background',
      type: 'IMG',
      // Assuming you've uploaded the corresponding asset
      src: assets('background.jpg'),
      meta: {
        // Metadata is any information that doesn't directly get used when rendering the video. The builder sets and uses certain metadata when it creates resources, like the 'size' property for images or videos, or the 'duration' property for audio and video resources. In the case of 'duration', the builder uses that property to determine if it has to loop the resource in order to keep it playing through all the automations that have been attached. Without that 'meta' property, the builder will not loop resources. Various layout / helper functions rely on these properties as well, so be careful that if you're adding your own resource, you add any metadata that helper functions rely on.
        size: { w: 1920, h: 1080 }
      },
      automation: [
        { t: 0, x: 960, y: 540, w: 1920, h: 1080, r: 0 },
        { t, x: 960, y: 540, w: 1920, h: 1080, r: 0 }
      ]
    }

    // Finally, your function should call 'this.buildResources', which will automatically ensure that all the automations are specified correctly, create additional media resources records by looping to fulfill the demands specified by the automations, or omit invalid resources (i.e. ones with no automation)
    return this.buildResources();
}

module.exports = Builder;