콘텐츠로 이동

Testing Strategy

Umbra 의 테스트 피라미드, 테스트 타입별 작성 지침, 커버리지 목표. 결제와 복구처럼 신뢰성 크리티컬한 영역은 더 엄격한 기준을 적용한다.

Why this guide exists

결제 SaaS 는 테스트 없이 운영할 수 없다. 그렇다고 100% 커버리지가 정답도 아니다. Umbra 는 "어느 영역에 얼마나 투자할지" 를 명시하여 효율을 높인다.

Test pyramid

flowchart TB
    E2E[" E2E 얇음 — 가장 비싼 테스트, 수 개만 ~5% "]
    INT[" Integration 중간 — 도메인 간 통합, DB/Redis 포함 ~25% "]
    UNIT[" Unit 두꺼움 — 빠르고 저렴, 대부분 여기 ~70% "]

    E2E --> INT
    INT --> UNIT

    style E2E fill:#fca5a5,stroke:#b91c1c,color:#000
    style INT fill:#fcd34d,stroke:#a16207,color:#000
    style UNIT fill:#86efac,stroke:#15803d,color:#000

비율

Unit 70%, Integration 25%, E2E 5%. 아래 coverage targets 와 함께 고려.

Coverage targets

Area Target Notes
Core domains (billing, licensing, recovery) 80% line coverage 복잡 + 신뢰성 크리티컬
Supporting (identity, guild, member, notification) 60% CRUD 성격, 주요 경로만
Generic (audit, webhook) 50% 단순 구조
Handlers (apps/api) 70% auth/validation 필수
Workflows (Temporal) 80% 결정성 / retry 경로 중요
Jobs (asynq) 60% handler 별 happy + error path

Line coverage 는 참고 지표. 크리티컬 path 의 behavioral coverage 가 더 중요.

Test types

Unit tests

가장 많은 수. 단일 함수/메서드를 mock 과 함께 검증.

대상:

  • 도메인 로직 (상태 전이, invariant)
  • Pure utility 함수
  • Handler (mock service 사용)
  • Workflow (Temporal test suite)

특징:

  • Millisecond 단위 실행
  • 병렬 실행 안전 (t.Parallel())
  • 외부 의존 없음 (DB, network)

Integration tests

도메인과 adapter 의 통합. 실제 DB / Redis / Temporal 사용.

대상:

  • 도메인 service → sqlc 쿼리 → DB 검증
  • Outbox publisher → poller → subscriber 전체 경로
  • Temporal workflow end-to-end (mock activity 최소)
  • API handler → service → DB (HTTP test)

특징:

  • Docker 의존 (testcontainers 또는 공유 컨테이너)
  • 실행 시간 수 초
  • testing.Short() 으로 skip 가능

E2E tests

실제 시스템에 가까운 전체 시나리오.

대상:

  • 봇 설치 → Free License → 구독 → Pro License → 복구 전체 플로우
  • 결제 실패 재시도 체인

특징:

  • 실제 4개 프로세스 (bot, api, worker, web) 기동
  • Docker Compose 기반
  • 수 분 단위 실행
  • CI 에서 nightly 또는 pre-release 만

MVP 단계에서는 E2E 수가 적음. Phase 2 에서 확대.

Stack

  • Unit — go test + table-driven, testify (assertions only)
  • Mock — 수동 interface 구현 (mockery 도입 검토)
  • Integration — Docker Compose + testcontainers-go (option)
  • HTTP — net/http/httptest, echo/test
  • DB — 실제 Postgres + trasaction rollback 격리
  • Temporal — temporal.io/sdk/testsuite
  • Frontend — Vitest + React Testing Library
  • E2E (Phase 2) — Playwright

Unit test 작성

Table-driven

func TestSubscription_AdvancePeriod(t *testing.T) {
    baseTime := time.Date(2026, 4, 1, 0, 0, 0, 0, time.UTC)

    cases := []struct {
        name            string
        initialCycle    int
        currentPeriodEnd time.Time
        newEnd          time.Time
        wantCycle       int
        wantStatus      string
        wantRetryCount  int
    }{
        {
            name:            "advances cycle and resets retry",
            initialCycle:    3,
            currentPeriodEnd: baseTime,
            newEnd:          baseTime.AddDate(0, 1, 0),
            wantCycle:       4,
            wantStatus:      "active",
            wantRetryCount:  0,
        },
        {
            name:            "first cycle",
            initialCycle:    0,
            currentPeriodEnd: baseTime,
            newEnd:          baseTime.AddDate(0, 1, 0),
            wantCycle:       1,
            wantStatus:      "active",
            wantRetryCount:  0,
        },
    }

    for _, tc := range cases {
        t.Run(tc.name, func(t *testing.T) {
            t.Parallel()

            sub := domain.Subscription{
                CycleCount:       tc.initialCycle,
                CurrentPeriodEnd: tc.currentPeriodEnd,
                Status:           "past_due",
                RetryCount:       3,
            }

            sub.AdvancePeriod(tc.newEnd, tc.newEnd.Add(time.Hour))

            if sub.CycleCount != tc.wantCycle {
                t.Errorf("CycleCount: got %d, want %d", sub.CycleCount, tc.wantCycle)
            }
            if string(sub.Status) != tc.wantStatus {
                t.Errorf("Status: got %s, want %s", sub.Status, tc.wantStatus)
            }
            if sub.RetryCount != tc.wantRetryCount {
                t.Errorf("RetryCount: got %d, want %d", sub.RetryCount, tc.wantRetryCount)
            }
        })
    }
}

Service test with mocks

func TestBillingService_StartSubscription_Success(t *testing.T) {
    subs := &mockSubscriptionRepo{}
    keys := &mockBillingKeyRepo{}
    attempts := &mockAttemptRepo{}
    toss := &mockTossClient{
        issueResult: &port.IssueResult{BillingKey: "bill_x", CardLast4: "1234"},
        chargeResult: &port.ChargeResult{
            PaymentKey: "pay_x",
            ApprovedAt: time.Date(2026, 4, 1, 12, 0, 0, 0, time.UTC),
        },
    }
    crypto := &mockCrypto{ciphertext: []byte("enc"), nonce: []byte("nonce")}
    events := &mockEventPublisher{}
    clock := &mockClock{now: time.Date(2026, 4, 1, 12, 0, 0, 0, time.UTC)}
    tx := &mockTxManager{}

    svc := app.NewService(subs, keys, attempts, toss, crypto, events, clock, tx, testLogger)

    sub, err := svc.StartSubscription(ctx, app.StartInput{
        PayerUserID:  userID,
        GuildID:      guildID,
        PlanCode:     "PRO",
        BillingKeyID: uuid.Nil,  // 새 빌링키 발급
        AuthKey:      "auth_x",
        CustomerKey:  "user_x",
    })

    require.NoError(t, err)
    assert.Equal(t, "active", string(sub.Status))
    assert.Equal(t, 1, sub.CycleCount)

    // 이벤트 검증
    assert.Len(t, events.published, 3)
    assert.Equal(t, "BillingKeyIssued", events.published[0].EventType())
    assert.Equal(t, "SubscriptionStarted", events.published[1].EventType())
    assert.Equal(t, "PaymentSucceeded", events.published[2].EventType())

    // Toss 호출 검증
    assert.Len(t, toss.chargeCalls, 1)
    assert.Equal(t, 9900, toss.chargeCalls[0].Amount)
}

Mock 작성

type mockSubscriptionRepo struct {
    inserted []domain.Subscription
    updated  []domain.Subscription
    getByIDFn func(ctx context.Context, id domain.SubscriptionID) (*domain.Subscription, error)
}

func (m *mockSubscriptionRepo) Insert(ctx context.Context, sub domain.Subscription) error {
    m.inserted = append(m.inserted, sub)
    return nil
}

func (m *mockSubscriptionRepo) Update(ctx context.Context, sub domain.Subscription) error {
    m.updated = append(m.updated, sub)
    return nil
}

func (m *mockSubscriptionRepo) GetByID(ctx context.Context, id domain.SubscriptionID) (*domain.Subscription, error) {
    if m.getByIDFn != nil { return m.getByIDFn(ctx, id) }
    return nil, domain.ErrSubscriptionNotFound
}
// ...

수동 mock 은 명시적이고 디버깅 쉬움. 복잡해지면 mockery 도입.

Integration test 작성

DB setup

// test/integration/setup.go
func NewTestPgxPool(t *testing.T) *pgxpool.Pool {
    t.Helper()

    dbURL := os.Getenv("TEST_DATABASE_URL")
    if dbURL == "" {
        t.Skip("TEST_DATABASE_URL not set")
    }

    pool, err := pgxpool.New(context.Background(), dbURL)
    require.NoError(t, err)

    t.Cleanup(func() {
        pool.Close()
    })

    return pool
}

트랜잭션 격리

각 테스트는 DB 상태 격리:

func TestBillingRepo_InsertAndGet(t *testing.T) {
    if testing.Short() { t.Skip() }

    pool := NewTestPgxPool(t)

    // 각 테스트는 별도 트랜잭션에서 시작, 마지막에 rollback
    tx, err := pool.Begin(context.Background())
    require.NoError(t, err)
    t.Cleanup(func() { tx.Rollback(context.Background()) })

    q := sqlc.New(tx)
    repo := subscriptionsqlc.NewRepo(q)

    sub := fixtures.Subscription()
    err = repo.Insert(ctx, sub)
    require.NoError(t, err)

    got, err := repo.GetByID(ctx, sub.ID)
    require.NoError(t, err)
    assert.Equal(t, sub.ID, got.ID)
}

Outbox 통합 테스트

func TestOutbox_EndToEnd(t *testing.T) {
    if testing.Short() { t.Skip() }

    pool := NewTestPgxPool(t)
    q := sqlc.New(pool)

    // Publisher: 이벤트 발행
    publisher := outbox.NewPublisher(q)
    err := publisher.Publish(ctx, testEvent)
    require.NoError(t, err)

    // Subscriber mock
    audit := &mockAuditConsumer{}
    licensing := &mockLicensingConsumer{}
    dispatcher := outbox.NewDispatcher(audit, licensing, ...)

    // Poller 수동 실행
    poller := outbox.NewPoller(q, dispatcher, testLogger)
    err = poller.ProcessBatch(ctx)
    require.NoError(t, err)

    // 구독자 호출 검증
    assert.Len(t, audit.events, 1)
    assert.Len(t, licensing.events, 1)

    // outbox published_at 설정 검증
    got, _ := q.GetOutboxEvent(ctx, eventID)
    assert.NotNil(t, got.PublishedAt)
}

Temporal workflow test

func TestRestoreWorkflow_Happy(t *testing.T) {
    ts := &testsuite.WorkflowTestSuite{}
    env := ts.NewTestWorkflowEnvironment()

    // Mock activities
    env.OnActivity(ValidateSnapshotActivity, mock.Anything, mock.Anything).
        Return(SnapshotData{...}, nil)
    env.OnActivity(PauseLiveSyncActivity, mock.Anything, mock.Anything).
        Return(nil)
    env.OnActivity(CreateOrUpdateRolesActivity, mock.Anything, mock.Anything).
        Return(nil)
    env.OnActivity(CreateOrUpdateChannelsActivity, mock.Anything, mock.Anything).
        Return(nil)
    env.OnActivity(ApplyPermissionOverridesActivity, mock.Anything, mock.Anything).
        Return(nil)
    env.OnActivity(ResumeLiveSyncActivity, mock.Anything, mock.Anything, mock.Anything).
        Return(nil)
    env.OnActivity(FinalizeRestoreJobActivity, mock.Anything, mock.Anything, mock.Anything).
        Return(nil)

    env.ExecuteWorkflow(RestoreWorkflow, RestoreInput{
        JobID:      jobID,
        SnapshotID: snapID,
        GuildID:    guildID,
        Scope:      []string{"roles", "channels", "permission_overrides"},
    })

    require.True(t, env.IsWorkflowCompleted())
    require.NoError(t, env.GetWorkflowError())

    // Activity 호출 순서 검증
    env.AssertActivityCalled(t, "ValidateSnapshotActivity")
    env.AssertActivityCalled(t, "PauseLiveSyncActivity")
    env.AssertActivityCalled(t, "ResumeLiveSyncActivity")
}

func TestRestoreWorkflow_ActivityFailsPermanently(t *testing.T) {
    ts := &testsuite.WorkflowTestSuite{}
    env := ts.NewTestWorkflowEnvironment()

    env.OnActivity(ValidateSnapshotActivity, mock.Anything, mock.Anything).
        Return(nil, temporal.NewNonRetryableApplicationError("invalid", "InvalidSnapshot", nil))

    env.ExecuteWorkflow(RestoreWorkflow, RestoreInput{...})

    require.True(t, env.IsWorkflowCompleted())
    require.Error(t, env.GetWorkflowError())

    // Resume 도 호출됐는지 (defer)
    // ...
}

HTTP handler test

func TestSubscriptionHandler_Start_Unauthorized(t *testing.T) {
    e := echo.New()
    req := httptest.NewRequest(http.MethodPost, "/api/v1/subscriptions",
        strings.NewReader(`{"guild_id":"018f...","plan_code":"PRO"}`))
    req.Header.Set("Content-Type", "application/json")
    rec := httptest.NewRecorder()
    c := e.NewContext(req, rec)

    h := &SubscriptionHandler{...}
    err := h.StartSubscription(c)

    // Error handler 거치기 전이라 domain error 그대로
    assert.ErrorIs(t, err, domain.ErrUnauthenticated)
}

func TestSubscriptionHandler_Start_Success(t *testing.T) {
    mockSvc := &mockBillingService{
        startResult: &domain.Subscription{
            ID:     subID,
            Status: "active",
        },
    }
    h := &SubscriptionHandler{billingSvc: mockSvc, logger: testLogger}

    // 세션 주입
    ctx := ctxpkg.WithUser(context.Background(), sess.User{ID: userID})

    e := echo.New()
    body := `{"guild_id":"018f...","plan_code":"PRO","auth_key":"x","customer_key":"y"}`
    req := httptest.NewRequest(http.MethodPost, "/api/v1/subscriptions", strings.NewReader(body)).
        WithContext(ctx)
    req.Header.Set("Content-Type", "application/json")
    rec := httptest.NewRecorder()
    c := e.NewContext(req, rec)

    err := h.StartSubscription(c)
    require.NoError(t, err)
    assert.Equal(t, 201, rec.Code)

    var resp SubscriptionResponse
    json.Unmarshal(rec.Body.Bytes(), &resp)
    assert.Equal(t, "active", resp.Status)
}

Frontend test

Component

import { render, screen } from "@testing-library/react"
import { SubscriptionCard } from "./SubscriptionCard"

describe("SubscriptionCard", () => {
  it("displays plan code", () => {
    const sub = { id: "1", status: "active" as const, planCode: "PRO", ... }
    render(<SubscriptionCard subscription={sub} />)
    expect(screen.getByText("PRO")).toBeInTheDocument()
  })

  it("shows cancel button for active subscription", () => {
    const sub = { id: "1", status: "active" as const, ... }
    render(<SubscriptionCard subscription={sub} />)
    expect(screen.getByRole("button", { name: /해지/i })).toBeInTheDocument()
  })
})

Hook with TanStack Query

import { renderHook, waitFor } from "@testing-library/react"
import { QueryClient, QueryClientProvider } from "@tanstack/react-query"
import { useSubscription } from "./useSubscription"

it("fetches subscription", async () => {
  const queryClient = new QueryClient()
  const wrapper = ({ children }) => (
    <QueryClientProvider client={queryClient}>{children}</QueryClientProvider>
  )

  // MSW 로 API mock
  server.use(rest.get("/api/v1/subscriptions/1", (req, res, ctx) =>
    res(ctx.json({ id: "1", status: "active", planCode: "PRO" }))
  ))

  const { result } = renderHook(() => useSubscription("1"), { wrapper })

  await waitFor(() => expect(result.current.isSuccess).toBe(true))
  expect(result.current.data.status).toBe("active")
})

Fixtures

// test/fixtures/billing.go
func Subscription(opts ...SubscriptionOpt) domain.Subscription {
    sub := domain.Subscription{
        ID:                domain.SubscriptionID(uuid.NewV7()),
        LicenseID:         domain.LicenseID(uuid.NewV7()),
        PayerUserID:       domain.UserID(uuid.NewV7()),
        GuildID:           domain.GuildID(uuid.NewV7()),
        BillingKeyID:      domain.BillingKeyID(uuid.NewV7()),
        PlanID:            domain.PlanID(uuid.NewV7()),
        Status:            "active",
        CurrentPeriodEnd:  time.Now().Add(30 * 24 * time.Hour),
        NextBillingAt:     time.Now().Add(30 * 24 * time.Hour),
        CycleCount:        1,
        RetryCount:        0,
    }
    for _, opt := range opts {
        opt(&sub)
    }
    return sub
}

type SubscriptionOpt func(*domain.Subscription)

func WithStatus(s string) SubscriptionOpt {
    return func(sub *domain.Subscription) { sub.Status = domain.Status(s) }
}

// 사용
sub := fixtures.Subscription(
    fixtures.WithStatus("past_due"),
    fixtures.WithRetryCount(2),
)

Functional options 패턴으로 유연성.

CI setup

GitHub Actions

name: CI
on: [pull_request, push]

jobs:
  test-go:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_USER: umbra
          POSTGRES_PASSWORD: umbra
          POSTGRES_DB: umbra_test
        ports: [5432:5432]
      redis:
        image: redis:7
        ports: [6379:6379]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with: { go-version: 1.23 }
      - run: make test-go
      - run: make test-integration
        env:
          TEST_DATABASE_URL: postgres://umbra:umbra@localhost:5432/umbra_test?sslmode=disable

  test-web:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: oven-sh/setup-bun@v1
      - run: bun install
      - run: bun run test

  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: golangci/golangci-lint-action@v4
      - run: bun run lint

PR gate

  • 모든 테스트 green
  • Coverage delta 가 크게 감소하면 alert (Phase 2)

Performance / Load test

Phase 2 에서 본격 도입:

  • k6 로 API load test
  • 동시 구독 100명 처리 검증
  • Recovery workflow 대형 길드 (10k 멤버) 시뮬레이션

MVP 는 실제 운영 데이터로 학습.

Do / Don't

Do

  • ✅ Core domain 은 table-driven test
  • ✅ Service test 에 모든 port mock
  • ✅ Integration test 는 transaction rollback 격리
  • ✅ Temporal workflow test with testsuite
  • ✅ Fixtures with functional options
  • ✅ HTTP test 로 auth/validation 경로 커버
  • ✅ t.Parallel() 적극 활용

Don't

  • ❌ 100% coverage 집착 (무의미한 테스트 추가)
  • ❌ Test-in-production (production DB 수정)
  • ❌ Flaky test 방치 (랜덤 sleep, race)
  • ❌ Shared global state (test 간 leak)
  • ❌ 실제 Toss API 호출 (sandbox 도 신중)
  • ❌ Mock 사용 남용 (integration 도 필요)

See also

  • backend-conventions.md
  • frontend-conventions.md
  • hexagonal-pattern.md
  • temporal-workflow.md
  • deployment.md