Golang Microservices Full Course #08 - Optimistic Locking & Idempotency Key

Main Ideas / Concepts Taught

• Optimistic locking to prevent race conditions in wallet/payment systems

  • The video focuses on concurrency issues in a wallet system where multiple requests can hit the same wallet “at the same time” (race conditions).
  • Example scenario (two operations on the same sender wallet happen in milliseconds):
    • Thread 1 reads balance = 100, then applies transfer out 70 → result would become 30.
    • Thread 2 reads balance = 100, then applies withdraw 50 → result would become 50.
  • Without concurrency control, both operations could succeed, effectively allowing spending more than the available balance—leading to incorrect money movement (the system “loses money”).
  • Correct enforcement should reject the second operation because:
    • 100 - 70 - 50 = -20

• Why optimistic locking vs pessimistic locking

  • Pessimistic locking
    • Locks the table/row until the transaction completes.
    • Safer, but can be slow because other threads must wait (queue/block).
  • Optimistic locking
    • Allows concurrent execution but uses a version field to detect conflicts.
    • Before updating, it checks that the version hasn’t changed.
    • If another transaction modified the row first, the version mismatch is detected and the transaction fails, so the client can retry.

• Using a wallet version column

  • Each wallet row has a version integer.
  • On successful update, the update increments the version (e.g., version = version + 1).
  • The update includes a condition: WHERE ... AND version = currentVersion.
  • If no rows are affected, it means the version changed due to a concurrent update:
    • the operation should fail with a concurrency conflict error.

• Idempotency via an “item potency key” (idempotency key)

  • The payment/transfer flow stores a transaction record keyed by an idempotency key (item_potency_key / “potency key”).
  • Before processing, the service checks if a transaction with that key already exists.
  • If it exists, the system stops and returns the existing transaction, preventing duplicate processing.

---

Methodology / Implementation Steps (Detailed)

A) Optimistic locking: update wallet balance safely

• Modify the Wallet repository to support transactional balance updates

  • Add a method like: UpdateBalanceTx(...) that:
    • Takes a transaction context: context and tx (SQL transaction).
    • Inputs:
      • walletID (string)
      • newBalance (float)
      • currentVersion (int)
    • Returns:
      • error if update fails (including version mismatch)

• SQL logic used

  • Execute an update like:
    • UPDATE wallets SET balance = ?, version = version + 1 WHERE id = ? AND version = ?

• Conflict detection

  • Check rowsAffected:
    • If rowsAffected == 0 → version mismatch → another concurrent update happened
    • Return an error indicating an optimistic/concurrency conflict so the client can retry.

• Execute within DB transaction

  • Use the tx passed in (from BEGIN/COMMIT/ROLLBACK flow).

---

B) Create transaction domain/models and repository

• Create a transaction folder structure under something like internal/transaction/:

  • model/tx.go (transaction entity)
  • repository/ (interface and MySQL implementation)
  • service/ (business logic)
  • handler/ (HTTP/business handler layer—mentioned as part of the folder)

• Transaction model includes fields

  • ID (string)
  • SenderWalletID (string, nullable in some flows like “top up”)
  • ReceiverWalletID (string, required)
  • Amount (float)
  • Description (string / JSON-like in subtitles)
  • ItemPotencyKey (string, required; used for idempotency)
  • Status (string; success implied)
  • CreatedAt (timestamp)

• There is also a transfer request structure:

  • ReceiverEmail (string, required with validation)
  • Amount (float, required, must be > 0)
  • Description (optional)
  • ItemPotencyKey (required, used for idempotency)

---

C) Transaction repository: idempotency lookup + insert

• Implement a repository with methods such as:
  • Create transaction record
    • INSERT INTO transactions (...) VALUES (...) using the transaction tx
  • Get transaction by idempotency key
    • SELECT ... FROM transactions WHERE item_potency_key = ?
    • If a row exists → it means the request was already processed.

---

D) Transaction service: the end-to-end transfer flow

• Start a DB transaction

  • Begin with BEGIN and ensure ROLLBACK on errors.
  • Commit at the end after all steps succeed.

• Idempotency check

  • Call: transactionRepo.GetByItemPotencyKey(ctx, key)
  • If found (existing != nil):
    • Return the existing transaction result immediately
    • Stop further processing (prevents double-processing).

• Load sender and receiver wallets

  • Fetch sender wallet by senderUserID
  • Fetch receiver wallet by receiverEmail (via user lookup → then wallet lookup)
  • Validations:
    • Receiver not found → return a custom “not found” style error
    • Sender wallet not found / receiver wallet not found → return appropriate errors
    • Sender and receiver cannot be the same account (self-transfer rejected)

• Validate available balance

  • If senderWallet.Balance < request.Amount → reject with “insufficient balance”.

• Apply optimistic locking updates

  • Compute newSenderBalance = senderWallet.Balance - amount
  • Call optimistic balance update for sender wallet:
    • walletRepo.UpdateBalanceTx(tx, senderWalletID, newSenderBalance, senderWallet.Version)
  • If optimistic locking fails (version mismatch):
    • Return a HTTP conflict / concurrency conflict error telling the client to retry.

• Update receiver wallet balance

  • Compute newReceiverBalance = receiverWallet.Balance + amount
  • Call optimistic balance update for receiver similarly (with version check)
  • If receiver update fails due to optimistic conflict:
    • Return the same style concurrency conflict error.

• Persist the transaction record

  • Create a new transaction with:
    • Sender wallet ID, receiver wallet ID, amount, description, item potency key, status=success, etc.
  • Insert using the repository within the same DB transaction.

• Write ledger entries (debit/credit)

  • Create ledger rows for both sides:
    • Debit entry for sender (entry type “debit”, amount = transfer amount)
    • Credit entry for receiver (entry type “credit”, amount = transfer amount)
  • Both entries are created within the same DB transaction.

• Commit

  • If all steps succeed:
    • COMMIT
    • Return the created transaction (and possibly updated wallet info as implied).

---

E) Error Handling Strategy (As Described)

Use custom errors / HTTP status codes conceptually:

• Conflict on optimistic locking version mismatch
  • e.g., “concurrency conflict / please retry the request”
• Not found (receiver/user/wallet missing)
  • return a not-found style error
• Insufficient balance
  • reject request
• Self-transfer
  • reject request

---

Speakers / Sources Featured

• No explicit speaker name is given.
• Source: a single course instructor speaking through the video (“Hello back to this course…”).