SiliconMark™ Benchmarks

​

Available Benchmarks

1. Quick Mark - A comprehensive single-node GPU performance test that always runs first
2. Cluster Network benchmark - Multi-node network connectivity and bandwidth testing
3. LLM Fine-Tuning benchmark - Single-node LLM fine-tuning performance benchmarks

​

QuickMark Benchmark

​

Overview

​

Configuration

​

Result Structure

{
  "test_results": [
    {
      "gpu_id": "GPU-000...",
      "bf16_tflops": 156.8,
      "fp16_tflops": 78.5,
      "fp32_tflops": 39.2,
      "fp32_cuda_core_tflops": 19.5,
      "mixed_precision_tflops": 312.4,
      "l2_bandwidth_gbs": 3890.5,
      "memory_bandwidth_gbs": 1935.4,
      "temperature_centigrade": 65,
      "power_consumption_watts": 350,
      "kernel_launch_overhead_us": 12.5,
      "device_to_host_bandwidth_gbs": 24.8,
      "host_to_device_bandwidth_gbs": 25.2
    },
    {
       "gpu_id": "GPU-111...",
       ...
    }
  ],
  "aggregate_results": {
    "total_fp16_tflops": 628.0,
    "total_fp32_tflops": 313.6,
    "avg_temperature": 67.2,
    "allreduce_bandwidth_gbs": 180.5,
    "broadcast_bandwidth_gbs": 175.3,
    "fp16_tflops_per_peak_watt": 1.79,
    "fp32_tflops_per_peak_watt": 0.89,
    "gpu_bandwidth_matrix": [
      {
        "source_gpu": 0,
        "target_gpu": 1,
        "duplex_gbs": 450.2,
        "simplex_gbs": 225.1,
        "connection_type": ...
      }
    ]
  }
}

​

Field Metadata

​

Cluster Network Benchmark

​

Overview

​

Result Structure (Cluster-level)

{
  "avg_bandwidth_gbps": 45.2,
  "avg_latency": 0.8,
  "min_bandwidth_gbps": 42.1,
  "total_links_tested": 12,
  "node_count": 4,
  "measurements": [
    {
      "host_ip": "192.168.1.10",
      "dest_ip": "192.168.1.11",
      "throughput_mbps": 45200,
      "throughput_gbps": 45.2,
      "latency_ms": 0.75
    }
  ]
}

​

Field Metadata

​

Llama3 Fine-tuning Single Node

​

Overview

​

Configuration

​

Fixed Parameters

• Sequence Length: 4096 tokens
• Micro Batch Size: 1 (optimized for packed sequences)
• Max Steps: 50 (default, configurable)
• Training Data: Synthetic (SquadDataModule)

​

Requirements

​

Software Requirements

• NeMo Container: nvcr.io/nvidia/nemo:24.12 will be downloaded automatically.
• HuggingFace Token: Required (set as HF_TOKEN environment variable). Get your token from https://huggingface.co/settings/tokens
• Disk Space Requirements:
  • 8B model: ~75GB (55GB base + 20GB model)
  • 70B model: ~205GB (55GB base + 150GB model)
  • 405B model: ~905GB (55GB base + 850GB model)

export HF_TOKEN="hf_xxxxxxxxxxxxxxxxxxxx"  # Required
export STAGE_PATH="$HOME/workspace/benchmark_stage"  # Optional

​

Hardware Requirements

• 8B LoRA FP8: ~14GB total (fits on single 40GB GPU)
• 70B LoRA FP8: ~60GB total (needs TP=2 on 40GB GPUs or single 80GB GPU)
• 405B LoRA FP8: ~315GB total (needs TP=8 on 80GB GPUs)

​

Parallelism Strategy

​

Auto-Parallelism Algorithm

1. Calculate Total Memory: Based on model size, dtype, and fine-tuning type
2. Determine Tensor Parallelism (TP):
   Copy

   min_TP = ceil(total_memory / gpu_memory)
   TP = nearest_power_of_2 ≥ min_TP
   
3. Calculate Data Parallelism (DP):
   Copy

   DP = total_gpus / TP
   
4. Set Global Batch Size (GBS):
   Copy

   GBS = DP * 2 (capped by model-specific limits)
   - 8B: max 64
   - 70B: max 32
   - 405B: max 16
   

​

Parallelism Components

1. Tensor Parallelism (TP)
   • Splits model layers across GPUs when model exceeds single GPU memory
   • Automatically calculated as smallest power of 2 that fits
   • Example: 70B model with FP16 (160GB) on 80GB GPUs requires TP=2
2. Data Parallelism (DP)
   • Uses remaining GPUs for parallel batch processing: DP = gpu_count / TP
   • Each DP replica processes different samples simultaneously
   • Example: 8 GPUs with TP=2 gives DP=4
3. Global Batch Size (GBS)
   • Total samples per training iteration
   • Auto-scaled based on DP with model-specific caps
   • Formula: GBS = min(DP * 2, model_cap)

​

Example Configurations

​

Result Structure

{
  "tokens_per_step": 16384,
  "tokens_per_second": 125000,
  "train_step_time_mean": 0.328,
  "train_step_time_std": 0.012,
  "step_time_cv_percent": 3.66,
  "time_to_1t_tokens_days": 92.6
}

​

Metrics Calculation

• Tokens per Step = global_batch_size × 4096
• Tokens per Second = tokens_per_step ÷ train_step_time_mean
• Time to 1T Tokens = 10^12 ÷ (tokens_per_second × 86400) days

​

Field Metadata

​

Configuration Example

{
  "model_size": "8b",
  "dtype": "fp8",
  "fine_tune_type": "lora",
}