Difference between revisions of "CSNG (File Format)"

From Retro Modding Wiki
Jump to: navigation, search
(→‎Header)
Line 1: Line 1:
 
The '''CSNG format''' contains MIDI data. It appears in Metroid Prime 1 and 2.  It is essentially MusyX's SON music format, with a custom header.  
 
The '''CSNG format''' contains MIDI data. It appears in Metroid Prime 1 and 2.  It is essentially MusyX's SON music format, with a custom header.  
−
 
−
{{research|3|Nothing is known about this format.}}
 
  
 
__TOC__
 
__TOC__
Line 7: Line 5:
 
== Format ==
 
== Format ==
  
−
All offsets are relative to the start of the main header (after the custom header).
+
All offsets are relative to the start of the main header (after the custom header).
 +
 
 +
Timings are represented in ''ticks'', like MIDI. Unlike MIDI, the tick-resolution is fixed at 384
 +
ticks per beat (e.g. 120 beats-per-minute works out to <code>384 * 120 / 60 = 768</code> ticks-per-second).
  
 
=== Custom Header ===
 
=== Custom Header ===
Line 55: Line 56:
 
| 0x4
 
| 0x4
 
| 4
 
| 4
−
| '''Channel Index Offset'''
+
| '''Track Index Offset'''; (absolute SON-offset)
 
|-
 
|-
 
| 0x8
 
| 0x8
 
| 4
 
| 4
−
| '''Channel Map Offset'''
+
| '''Channel Map Offset'''; (absolute SON-offset)
 
|-
 
|-
 
| 0xC
 
| 0xC
 
| 4
 
| 4
−
| '''Tempo Table Offset''', 0x0 if tempo doesn't change
+
| '''Tempo Table Offset'''; (absolute SON-offset) 0x0 if tempo doesn't change
 
|-
 
|-
 
| 0x10
 
| 0x10
Line 75: Line 76:
 
| 0x18
 
| 0x18
 
| 256
 
| 256
−
| '''Channel Offsets'''; 64 elements
+
| '''Track Header Offsets'''; (absolute SON-offsets) 64 elements, 0x0 if track not present
 
|-
 
|-
 
| 0x118
 
| 0x118
Line 81: Line 82:
 
|}
 
|}
  
−
==Data==
+
===Track Header===
  
−
The data in the files is very similar to MIDI, however it is formatted by word, rather than MIDI's byte formatting.
+
This is a variable-length table of headers for each track
  
−
After the data for the instruments are defined, there is a table of offsets for data through out the SON data.  Each chunk of data starts with a long 0x8, usually followed by another absolute offset that points to the end of the chunk.
+
{| class="wikitable"
 +
! Offset
 +
! Size
 +
! Description
 +
|-
 +
| 0x0
 +
| 4
 +
| '''Start Tick'''; time-point to begin executing track data  
 +
|-
 +
| 0x4
 +
| 4
 +
| {{unknown|'''Unknown'''}}; commonly 0xffff0000
 +
|-
 +
| 0x8
 +
| 2
 +
| '''Track Data Index'''
 +
|-
 +
| 0xA
 +
| 2
 +
| '''Padding'''
 +
|-
 +
| 0xC
 +
| 4
 +
| '''Start Tick'''; copy of start tick
 +
|-
 +
| 0x10
 +
| 4
 +
| {{unknown|'''Unknown'''}}; commonly 0xffff0000
 +
|-
 +
| 0x14
 +
| 4
 +
| {{unknown|'''Unknown'''}}; commonly 0xffff0000
 +
|-
 +
| 0x18
 +
| colspan=2 {{unknown|End of header}}
 +
|}
  
−
Notes durations are controlled by 96 ticks per beat.
+
===Track Data===
 +
 
 +
Here begins a free-form blob of indexed track data. It starts with a variable-length
 +
'''u32 array''' of SON offsets for each track, then the track data itself.
 +
 
 +
====Track Data Header====
 +
 
 +
{| class="wikitable"
 +
! Offset
 +
! Size
 +
! Description
 +
|-
 +
| 0x0
 +
| 4
 +
| '''Track Data Header Size'''; size of the header ''after'' this field (always 0x8)
 +
|-
 +
| 0x4
 +
| 4
 +
| '''Pitch Wheel Data Offset'''; (absolute SON-offset) 0x0 if no pitch-wheel messages on track
 +
|-
 +
| 0x8
 +
| 4
 +
| '''Mod Wheel Data Offset'''; (absolute SON-offset) 0x0 if no mod-wheel messages on track
 +
|-
 +
| 0xC
 +
| colspan=2 {{unknown|End of header}}
 +
|}
 +
 
 +
====Track Commands====
 +
 
 +
After the track data header, the actual playback commands begin. There are only 2 types of commands
 +
in SON: ''note'' and ''control change''.
 +
 
 +
=====Delta Time RLE=====
 +
 
 +
Just like MIDI, each command starts with a '''delta time''' value telling the sequencer
 +
how many ticks to wait after the previous command. Unlike MIDI, Factor5 uses a custom
 +
[[wikipedia:Run-length encoding|RLE scheme]] to adaptively scale the value's precision
 +
to reduce the value's size.
 +
 
 +
The RLE operates on 16-bit words, with the value 0xffff triggering a continuation,
 +
then a 'dead' 16-bit word skipped over, then the 0xffff is summed with the following RLE value,
 +
looping the decode algorithm.
 +
 
 +
In Python, decoding works like so:
 +
 
 +
<syntaxhighlight lang="python" line="1">
 +
def DecodeDeltaTimeRLE(in):
 +
    total = 0
 +
    while True:
 +
        term = in.ReadU16()
 +
        if term == 0xffff:
 +
            total += 0xffff
 +
            dummy = in.ReadU16()
 +
            continue
 +
        total += term
 +
        return total
 +
</syntaxhighlight>
 +
 
 +
=====Note Command=====
 +
 
 +
When the two bytes following the delta-time != 0xffff, and the high-bit of the first byte is ''set'',
 +
this is a '''note command'''.
 +
 
 +
Unlike MIDI, which has separate commands for note-on/note-off, SON attaches a ''note length'' value
 +
to a note-on command, which is then able to track its own lifetime.
 +
 
 +
{| class="wikitable"
 +
! Offset
 +
! Size
 +
! Description
 +
|-
 +
| 0x0
 +
| 1
 +
| '''Note'''; AND with 0x7f for the value
 +
|-
 +
| 0x1
 +
| 1
 +
| '''Velocity'''; AND with 0x7f for the value
 +
|-
 +
| 0x2
 +
| 2
 +
| '''Note Length'''; count of ticks before note-off issued by sequencer
 +
|-
 +
| 0x4
 +
| colspan=2 {{unknown|End of note}}
 +
|}
 +
 
 +
=====Control Change Command=====
 +
 
 +
When the two bytes following the delta-time != 0xffff, and the high-bit of the first byte is ''unset'',
 +
this is a '''control change command'''.
 +
 
 +
{| class="wikitable"
 +
! Offset
 +
! Size
 +
! Description
 +
|-
 +
| 0x0
 +
| 1
 +
| '''Value'''; AND with 0x7f for the value
 +
|-
 +
| 0x1
 +
| 1
 +
| '''Control'''; AND with 0x7f for the value
 +
|-
 +
| 0x2
 +
| colspan=2 {{unknown|End of control change}}
 +
|}
 +
 
 +
=====End Of Track=====
 +
 
 +
When the two bytes following the delta-time == 0xffff, this track has no more commands.
 +
 
 +
====Continuous Pitch / Modulation Data====
 +
 
 +
If the pitch or mod offsets in a track are non-zero, they point to a buffer of RLE-compressed
 +
(delta-tick, delta-value) pairs, decoding to signed 16-bit precision. The decoder must track
 +
the absolute time and value, summing each consecutive update for the current time/values.
 +
 
 +
The algorithm for this RLE is different than the delta-time one for commands. It may
 +
scale down to a single byte if able.
 +
 
 +
<syntaxhighlight lang="python" line="1">
 +
def DecodeRLE(in):
 +
    term = in.ReadU8()
 +
    total = term & 0x7f
 +
    if term & 0x80:
 +
        total *= 256 + in.ReadU8()
 +
    return total
 +
 
 +
def DecodeContinuousRLE(in):
 +
    total = 0
 +
    while True:
 +
        term = DecodeRLE(in)
 +
        if term == 0x8000:
 +
            total += 0xffff
 +
            dummy = in.ReadU8()
 +
            continue
 +
        total += term
 +
 
 +
        if total >= 0x4000:
 +
            return total - 0xffff
 +
        else:
 +
            return total
 +
</syntaxhighlight>
 +
 
 +
===Channel Map===
 +
 
 +
This is a simple '''u8 table''' mapping 64 SON tracks to 16 MIDI channels for instrument selection via the [[AGSC (File Format)#MIDI Setup Entry|SongGroup MIDI-Setup]].
 +
 
 +
===Tempo Table===
 +
 
 +
When the SON has a non-zero tempo table offset, this song features tempo changes.
 +
The change events are simple absolute-tick / BPM pairs.
 +
 
 +
{| class="wikitable"
 +
! Offset
 +
! Size
 +
! Description
 +
|-
 +
| 0x0
 +
| 4
 +
| '''Tick'''; absolute time-point to perform tempo change
 +
|-
 +
| 0x4
 +
| 4
 +
| '''Tempo'''; new tempo in BPM
 +
|-
 +
| 0x2
 +
| colspan=2 {{unknown|End of tempo change}}
 +
|}
  
 
[[Category:File Formats]]
 
[[Category:File Formats]]
 
[[Category:Metroid Prime]]
 
[[Category:Metroid Prime]]
 
[[Category:Metroid Prime 2: Echoes]]
 
[[Category:Metroid Prime 2: Echoes]]

Revision as of 14:35, 19 May 2016

The CSNG format contains MIDI data. It appears in Metroid Prime 1 and 2. It is essentially MusyX's SON music format, with a custom header.

Format

All offsets are relative to the start of the main header (after the custom header).

Timings are represented in ticks, like MIDI. Unlike MIDI, the tick-resolution is fixed at 384 ticks per beat (e.g. 120 beats-per-minute works out to 384 * 120 / 60 = 768 ticks-per-second).

Custom Header

This 0x14-byte header isn't part of the MusyX format; it appears at the start of the file. After parsing this the rest of the file is copied into a buffer and then passed to the MusyX functions.

Offset Size Description
0x0 4 Magic; (always 0x2)
0x4 4 MIDI Setup ID
0x8 4 SongGroup ID
0xC 4 AGSC ID
0x10 4 SON File Length
0x14 MusyX data starts

Header

Offset Size Description
0x0 4 Version; always 0x18
0x4 4 Track Index Offset; (absolute SON-offset)
0x8 4 Channel Map Offset; (absolute SON-offset)
0xC 4 Tempo Table Offset; (absolute SON-offset) 0x0 if tempo doesn't change
0x10 4 Initial Tempo; (commonly 0x78 or 120 beats per minute)
0x14 4 Unknown
0x18 256 Track Header Offsets; (absolute SON-offsets) 64 elements, 0x0 if track not present
0x118 End of header

Track Header

This is a variable-length table of headers for each track

Offset Size Description
0x0 4 Start Tick; time-point to begin executing track data
0x4 4 Unknown; commonly 0xffff0000
0x8 2 Track Data Index
0xA 2 Padding
0xC 4 Start Tick; copy of start tick
0x10 4 Unknown; commonly 0xffff0000
0x14 4 Unknown; commonly 0xffff0000
0x18 End of header

Track Data

Here begins a free-form blob of indexed track data. It starts with a variable-length u32 array of SON offsets for each track, then the track data itself.

Track Data Header

Offset Size Description
0x0 4 Track Data Header Size; size of the header after this field (always 0x8)
0x4 4 Pitch Wheel Data Offset; (absolute SON-offset) 0x0 if no pitch-wheel messages on track
0x8 4 Mod Wheel Data Offset; (absolute SON-offset) 0x0 if no mod-wheel messages on track
0xC End of header

Track Commands

After the track data header, the actual playback commands begin. There are only 2 types of commands in SON: note and control change.

Delta Time RLE

Just like MIDI, each command starts with a delta time value telling the sequencer how many ticks to wait after the previous command. Unlike MIDI, Factor5 uses a custom RLE scheme to adaptively scale the value's precision to reduce the value's size.

The RLE operates on 16-bit words, with the value 0xffff triggering a continuation, then a 'dead' 16-bit word skipped over, then the 0xffff is summed with the following RLE value, looping the decode algorithm.

In Python, decoding works like so: 

  1. def DecodeDeltaTimeRLE(in):
  2.     total = 0
  3.     while True:
  4.         term = in.ReadU16()
  5.         if term == 0xffff:
  6.             total += 0xffff
  7.             dummy = in.ReadU16()
  8.             continue
  9.         total += term
  10.         return total
Note Command

When the two bytes following the delta-time != 0xffff, and the high-bit of the first byte is set, this is a note command.

Unlike MIDI, which has separate commands for note-on/note-off, SON attaches a note length value to a note-on command, which is then able to track its own lifetime.

Offset Size Description
0x0 1 Note; AND with 0x7f for the value
0x1 1 Velocity; AND with 0x7f for the value
0x2 2 Note Length; count of ticks before note-off issued by sequencer
0x4 End of note
Control Change Command

When the two bytes following the delta-time != 0xffff, and the high-bit of the first byte is unset, this is a control change command.

Offset Size Description
0x0 1 Value; AND with 0x7f for the value
0x1 1 Control; AND with 0x7f for the value
0x2 End of control change
End Of Track

When the two bytes following the delta-time == 0xffff, this track has no more commands.

Continuous Pitch / Modulation Data

If the pitch or mod offsets in a track are non-zero, they point to a buffer of RLE-compressed (delta-tick, delta-value) pairs, decoding to signed 16-bit precision. The decoder must track the absolute time and value, summing each consecutive update for the current time/values.

The algorithm for this RLE is different than the delta-time one for commands. It may scale down to a single byte if able. 

  1. def DecodeRLE(in):
  2.     term = in.ReadU8()
  3.     total = term & 0x7f
  4.     if term & 0x80:
  5.         total *= 256 + in.ReadU8()
  6.     return total
  7.  
  8. def DecodeContinuousRLE(in):
  9.     total = 0
  10.     while True:
  11.         term = DecodeRLE(in)
  12.         if term == 0x8000:
  13.             total += 0xffff
  14.             dummy = in.ReadU8()
  15.             continue
  16.         total += term
  17.  
  18.         if total >= 0x4000:
  19.             return total - 0xffff
  20.         else:
  21.             return total

Channel Map

This is a simple u8 table mapping 64 SON tracks to 16 MIDI channels for instrument selection via the SongGroup MIDI-Setup.

Tempo Table

When the SON has a non-zero tempo table offset, this song features tempo changes. The change events are simple absolute-tick / BPM pairs.

Offset Size Description
0x0 4 Tick; absolute time-point to perform tempo change
0x4 4 Tempo; new tempo in BPM
0x2 End of tempo change
Retrieved from "http://www.metroid2002.com/retromodding/index.php?title=CSNG_(File_Format)&oldid=1454"