Announcing StickerBaker!

Make stickers with AI. Powered by @replicate and @flydotio, and 100% open-source.https://t.co/8vucCsHtAd pic.twitter.com/tBhDyGrOx0

— Charlie Holtz (@charliebholtz) February 26, 2024

Enter a prompt and generating a sticker using https://replicate.com/fofr/sticker-maker.

Here's an overview of the architecture:

The home page is rendered in lib/sticker_web/home_live.ex. When the prompt form is submitted, this handle_event gets called:

  def handle_event("save", %{"prompt" => prompt}, socket) do
    user_id = socket.assigns.local_user_id

    {:ok, prediction} =
      Predictions.create_prediction(%{
        prompt: prompt,
        local_user_id: user_id
      })

    send(self(), {:kick_off, prediction})

    {:noreply,
     socket
     |> assign(form: to_form(%{"prompt" => ""}))
     |> stream_insert(:my_predictions, prediction, at: 0)}
  end

This sends a :kick_off message to the LiveView (so there is no lag) which calls Predictions.moderate/3 in lib/sticker/predictions.ex:

  @doc """
  Moderates a prediction.
  The logic in replicate_webhook_controller.ex handles
  the webhook. Once the moderation is complete, the webhook controller automatically
  called gen_image.
  """
  def moderate(prompt, user_id, prediction_id) do
    "fofr/prompt-classifier"
    |> Replicate.Models.get!()
    |> Replicate.Models.get_latest_version!()
    |> Replicate.Predictions.create(
      %{
        prompt: "[PROMPT] #{prompt} [/PROMPT] [SAFETY_RANKING]",
        max_new_tokens: 128,
        temperature: 0.2,
        top_p: 0.9,
        top_k: 50,
        stop_sequences: "[/SAFETY_RANKING]"
      },
      "#{Sticker.Utils.get_host()}/webhooks/replicate?user_id=#{user_id}&prediction_id=#{prediction_id}"
    )
  end

We pass a webhook to Replicate. All the logic for the webhook lives in lib/sticker_web/controllers/replicate_webhook_controller.ex. The nice thing about this webhook is that we can refresh the page or disconnect and Replicate still handles the prediction queue for us. Once the prediction is ready, we upload it to Tigris (Replicate doesn't save our data for us) and then the sticker gets broadcast back to our home_live.ex.

Importantly, because we're passing Replicate a webhook, for local dev you'll need ngrok running to tunnel your localhost to a URL. Once you install ngrok run it with ngrok http 4000 and paste the URL into your copied .env file.

StickerBaker runs on:

• Replicate to generate the stickers
• Fly.io for infrastructure
• Tigris for image hosting

To start your Phoenix server:

• Run mix setup to install and setup dependencies
• Create an env file with cp .env.copy .env
  • Add your Replicate tokens
  • Add Tigris tokens
  • Start ngrok with ngrok http 4000 and add that to your env
• Start Phoenix endpoint with mix phx.server or inside IEx with iex -S mix phx.server
• Add a .env file with REPLICATE_API_TOKEN set to your Replicate token

Now you can visit localhost:4000 from your browser.

Update the url and check_origin origin in prod.exs Deploy with fly launch Make sure when you fly launch you set up a Postgres DB!