↓ Skip to main content
  1. Programming Languages/
  2. Cloud-Native Golang/

Mastering Go JSON: Custom Marshaling & High-Performance Optimization

·1941 words·10 mins·
Go Advanced Performance Web Development Golang Json Performance Marshalling Optimization Backend Simd
Jeff Taakey
Author
Jeff Taakey
21+ Year CTO & Multi-Cloud Architect. Bridging the gap between theoretical CS and production-grade engineering for 300+ deep-dive guides.
Table of Contents

Introduction
#

In the landscape of modern backend development, JSON is the lingua franca. Whether you are building microservices communicating via gRPC-Gateway, RESTful APIs, or event-driven systems processing Kafka messages, your Go application is likely spending a significant amount of CPU cycles serializing and deserializing JSON data.

As we move through 2025, the standard encoding/json library in Go remains robust and correct, but it isn’t always the fastest tool in the shed. For high-throughput applications—think tens of thousands of requests per second—the reflection-heavy nature of the standard library can become a bottleneck.

But performance isn’t the only concern. Often, business logic dictates data representation that doesn’t map 1-to-1 with your struct fields. You might need to mask PII (Personally Identifiable Information), handle polymorphic types, or deal with legacy API formats that send numbers as strings.

In this deep dive, we are going to move beyond the basics. We will explore how to take full control of JSON encoding with custom marshaler interfaces, and then we will shift gears to raw performance, comparing the standard library against modern, high-performance alternatives like goccy/go-json and segmentio/encoding.

What you will learn:

  1. How to implement MarshalJSON and UnmarshalJSON safely (avoiding the recursion trap).
  2. Techniques for handling dynamic JSON and mixed types.
  3. Understanding the cost of Reflection vs. Code Generation vs. SIMD.
  4. Real-world benchmarking of different JSON strategies.

Prerequisites
#

To follow along with the code examples and benchmarks, ensure you have the following setup:

  • Go Version: Go 1.23 or higher (recommended for the latest compiler optimizations).
  • IDE: VS Code (with Go extension) or GoLand.
  • Terminal: A standard shell to run go test and benchmarks.

Initialize a new project to keep your workspace clean:

mkdir go-json-deepdive
cd go-json-deepdive
go mod init github.com/yourname/go-json-deepdive

Part 1: The Standard Library & Custom Marshaling
#

The encoding/json package uses Go’s reflect package to inspect struct tags and fields at runtime. While convenient, this flexibility comes at a cost. However, before optimizing for speed, we must optimize for correctness and utility.

The Marshaler and Unmarshaler Interfaces
#

Go allows types to customize their own JSON representation by implementing these two interfaces:

// Marshaler interface
type Marshaler interface {
    MarshalJSON() ([]byte, error)
}

// Unmarshaler interface
type Unmarshaler interface {
    UnmarshalJSON([]byte) error
}

If a type implements these, encoding/json bypasses the default reflection logic and calls these methods directly.

Scenario: The “Secure User” Problem
#

Imagine you have a User struct. In your database, you store a password hash and a precise creation timestamp. However, when sending this data to a frontend client, you need to:

  1. Redact the sensitive fields completely.
  2. Format the timestamp to a specific ISO layout.
  3. Rename fields dynamically based on context (though struct tags usually handle renaming, logic might be required).

The Recursive Trap
#

A common mistake developers make when implementing custom marshaling is creating an infinite loop.

type User struct {
    ID        int       `json:"id"`
    Email     string    `json:"email"`
    Password  string    `json:"password"` // Should be hidden
    CreatedAt time.Time `json:"created_at"`
}

// BAD IMPLEMENTATION - DO NOT USE
func (u User) MarshalJSON() ([]byte, error) {
    // This calls json.Marshal, which sees the User type, 
    // which calls MarshalJSON again... Stack Overflow!
    return json.Marshal(u)
}

The Solution: Type Aliasing
#

To reuse the default behavior for most fields while customizing others, we use an Alias. An alias inherits the structure of the type but not its methods.

Create a file named custom_marshal.go:

package main

import (
	"encoding/json"
	"fmt"
	"log"
	"time"
)

type User struct {
	ID        int       `json:"id"`
	Email     string    `json:"email"`
	Password  string    `json:"password"` // Sensitive
	CreatedAt time.Time `json:"created_at"`
}

// Custom MarshalJSON to hide Password and format Time
func (u User) MarshalJSON() ([]byte, error) {
	// 1. Define an alias to prevent infinite recursion
	type Alias User

	// 2. Create an anonymous struct that embeds the Alias.
	// This allows us to override specific fields.
	return json.Marshal(&struct {
		Password string `json:"password,omitempty"` // Override to hide
		LastActive string `json:"last_active"`      // Add new calculated field
		*Alias                                      // Embed the rest
	}{
		Password:   "", // Explicitly empty (combined with omitempty removes it)
		LastActive: u.CreatedAt.Format(time.Kitchen), // Custom format
		Alias:      (*Alias)(&u),
	})
}

func main() {
	u := User{
		ID:        101,
		Email:     "[email protected]",
		Password:  "hashed_secret_123",
		CreatedAt: time.Now(),
	}

	data, err := json.MarshalIndent(u, "", "  ")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(string(data))
}

Output:

{
  "password": "",
  "last_active": "3:04PM",
  "id": 101,
  "email": "[email protected]",
  "created_at": "2025-10-24T15:04:05.123456-07:00"
}

Note: Depending on how strict omitempty is and the JSON library version, explicitly setting the field to an empty string might still show the key "". If you want to completely remove the key, simply don’t include the Password field in the anonymous struct wrapper.

Handling Polymorphic JSON (Custom Unmarshaling)
#

A more complex scenario involves “Polymorphic” JSON, where a field can be one of several types depending on another field (e.g., a type field in an event payload).

type Event struct {
	Type    string          `json:"type"`
	Payload json.RawMessage `json:"payload"` // Delay parsing
}

type LoginPayload struct {
	Username string `json:"username"`
}

type PurchasePayload struct {
	Amount int `json:"amount"`
	Item   string `json:"item"`
}

func ProcessEvent(jsonData []byte) {
	var e Event
	if err := json.Unmarshal(jsonData, &e); err != nil {
		log.Println("Error parsing event wrapper:", err)
		return
	}

	switch e.Type {
	case "login":
		var p LoginPayload
		json.Unmarshal(e.Payload, &p)
		fmt.Printf("Login Event: %s\n", p.Username)
	case "purchase":
		var p PurchasePayload
		json.Unmarshal(e.Payload, &p)
		fmt.Printf("Purchase Event: %s for $%d\n", p.Item, p.Amount)
	}
}

Using json.RawMessage is a powerful technique. It defers the decoding cost of the inner payload until you know what schema to apply.

Part 2: The Performance Bottleneck
#

Why is standard encoding/json considered slow?

  1. Reflection: It has to inspect types at runtime.
  2. Allocation: It creates significant garbage (interface wrappers, intermediate buffers).
  3. Scanning: It reads the JSON byte-by-byte in a generic way.

Below is a visualization of the flow difference between standard reflection and optimized approaches.

flowchart LR subgraph "Standard Library (Reflection)" A[Input JSON] --> B{Inspect Type via Reflect} B --> C[Analyze Struct Tags] C --> D[Allocate Generic Value] D --> E[Parse & Assign] end subgraph "Optimized Libs (CodeGen / SIMD)" F[Input JSON] --> G[Pre-compiled Decoder / Assembly] G --> H[Direct Memory Assignment] H --> I[Zero-Allocation Slicing] end style Standard fill:#f9f,stroke:#333,stroke-width:2px style Optimized fill:#bbf,stroke:#333,stroke-width:2px

Profiling the Issue
#

Before optimizing, we must prove the bottleneck. Let’s write a benchmark.

Create bench_test.go:

package main

import (
	"encoding/json"
	"testing"
)

type BigData struct {
	ID          string   `json:"id"`
	Name        string   `json:"name"`
	Description string   `json:"description"`
	Tags        []string `json:"tags"`
	Attributes  map[string]interface{} `json:"attributes"`
	Active      bool     `json:"active"`
}

func generateData() BigData {
	return BigData{
		ID:          "uuid-1234-5678",
		Name:        "Optimizing JSON in Go",
		Description: "A very long description that simulates real world payloads...",
		Tags:        []string{"golang", "json", "performance", "backend"},
		Attributes:  map[string]interface{}{"score": 99.5, "verified": true},
		Active:      true,
	}
}

func BenchmarkStdLibMarshal(b *testing.B) {
	data := generateData()
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		_, _ = json.Marshal(&data)
	}
}

func BenchmarkStdLibUnmarshal(b *testing.B) {
	data := generateData()
	bytes, _ := json.Marshal(&data)
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		var out BigData
		_ = json.Unmarshal(bytes, &out)
	}
}

Run the benchmark:

go test -bench=. -benchmem

You will likely see results indicating a noticeable amount of B/op (bytes per operation) and allocs/op. This is “garbage” that the GC has to clean up later.

Part 3: High-Performance Alternatives
#

In 2025, the Go ecosystem has several mature contenders for JSON processing.

1. segmentio/encoding
#

This is largely a drop-in replacement for the standard library. It doesn’t require code generation and is generally 2x-3x faster.

2. easyjson (Code Generation)
#

easyjson generates Go code for marshaling/unmarshaling.

  • Pros: Extremely fast, no reflection.
  • Cons: Requires a build step (easyjson -all struct.go), increases binary size, slightly harder developer experience.

3. goccy/go-json (The Modern King)
#

This library is gaining massive traction. It uses SIMD (Single Instruction, Multiple Data) assembly instructions (AVX2/NEON) to parse JSON at blazing speeds. It is a drop-in replacement and often beats code-generation libraries without the build-step hassle.

Comparison Table
#

Feature Standard Lib EasyJSON Segmentio Goccy/go-json
Approach Reflection Code Generation Optimized Reflection JIT / SIMD Assembly
Setup Built-in Extra build step Drop-in Drop-in
Marshaling Speed Baseline (1x) ~3x-4x ~2x ~3x-5x
Unmarshaling Speed Baseline (1x) ~3x-4x ~2x ~4x-6x
Memory Usage High Low Medium Low
Compatibility 100% High High Very High

Implementing goccy/go-json
#

Let’s modify our benchmark to include goccy/go-json.

First, install it:

go get github.com/goccy/go-json

Update bench_test.go:

import (
	"testing"
	"encoding/json"
	goccy "github.com/goccy/go-json"
)

// ... existing code ...

func BenchmarkGoccyMarshal(b *testing.B) {
	data := generateData()
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		_, _ = goccy.Marshal(&data)
	}
}

func BenchmarkGoccyUnmarshal(b *testing.B) {
	data := generateData()
	bytes, _ := goccy.Marshal(&data)
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		var out BigData
		_ = goccy.Unmarshal(bytes, &out)
	}
}

When you run this benchmark, the results are usually staggering. goccy can process hundreds of megabytes per second more than the standard library.

Part 4: Advanced Optimization Techniques
#

Even with a fast library, how you use it matters.

1. Zero-Allocation (Streaming)
#

If you are processing a massive JSON file (e.g., a 2GB export from a database), reading the whole file into memory ( ioutil.ReadFile ) and then Unmarshaling it is a recipe for an OOM (Out Of Memory) crash.

Use json.Decoder for streaming.

func ProcessLargeFile(r io.Reader) error {
    decoder := json.NewDecoder(r)
    
    // Check the opening delimiter
    if _, err := decoder.Token(); err != nil {
        return err
    }

    // While there are more items in the array
    for decoder.More() {
        var user User
        // Decode one item at a time
        if err := decoder.Decode(&user); err != nil {
            return err
        }
        // Process user immediately (e.g., insert to DB)
        // User struct is reused or GC'd quickly
    }

    return nil
}

2. Object Recycling with sync.Pool
#

If your high-throughput server creates thousands of User structs per second just to discard them after serialization, you are pressuring the Garbage Collector.

You can reuse structs using sync.Pool.

var userPool = sync.Pool{
    New: func() interface{} {
        return &User{}
    },
}

func HandleRequest(data []byte) {
    // Get from pool
    u := userPool.Get().(*User)
    
    // IMPORTANT: Reset fields to zero-values
    u.ID = 0
    u.Email = ""
    // ... reset other fields
    
    _ = json.Unmarshal(data, u)
    
    // Do logic...
    
    // Return to pool
    userPool.Put(u)
}

Warning: Only use sync.Pool if you have profiled your application and identified GC as a major bottleneck. Improper implementation can lead to subtle bugs (data leaks between requests).

3. Pre-computing JSON for Static Fields
#

If parts of your JSON response never change (e.g., status codes, API version headers), don’t marshal them every time.

var successHeader = []byte(`{"status":"success","version":"v1","data":`)
var jsonFooter = []byte(`}`)

func WriteResponse(w http.ResponseWriter, payload interface{}) {
    w.Write(successHeader)
    json.NewEncoder(w).Encode(payload)
    w.Write(jsonFooter) // Note: Encode usually adds a newline, be careful with syntax
}

Note: The above is a naive concatenation example. In production, ensure valid JSON syntax (handling trailing commas, etc.).

Conclusion
#

JSON performance in Go is a journey. Start with the standard library (encoding/json) because it is stable and bug-free.

  1. Custom Logic: Use MarshalJSON with the “Type Alias” trick to handle specific formatting or masking requirements without reflection recursion.
  2. Profiling: Before optimizing speed, use go test -bench to measure allocations.
  3. Library Switch: For an immediate performance boost in 2025, switching to goccy/go-json is often the highest ROI action you can take—it offers SIMD speeds with zero code changes.
  4. Architecture: For massive data, prefer streaming (Decoder) over loading full blobs into memory.

As we build more data-intensive applications, these milliseconds add up to substantial cloud cost savings and better user experiences.

Further Reading
#

Have you encountered a weird JSON edge case in Go? Let me know in the comments below or hit me up on Twitter!

Related Articles