General/sharplib
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/mh/core' into mh/dev

Browse Source 
# Conflicts:
#	archive/logging/GC.cs
#	archive/logging/Tracing.cs
This commit is contained in:
Marc Hernandez 2026-03-18 10:13:55 -07:00
commit cc52660989
84 changed files with 857 additions and 7946 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

93
AGENTS.md Normal file
View File

@ -0,0 +1,93 @@
# SharpLib and general coding guidelines
## 1. CORE PHILOSOPHY
* **Leaky Abstractions:** All abstractions are always leaky. Do not hide complexity. Expose failure modes, costs, and "why" parameters.
* **Visual Cost:** "Expensive things must look expensive." No hidden RPCs. Use explicit Message structs and `Send` methods.
* **Fast & Introspectable:** Code must be highly performant (hot-path aware) but deeply debuggable (`log`, `DebugOld`, `reason` strings).
## 2. NAMING & SEMANTICS
* **Leaf Libraries:** Core libs (`io`, `imm`, `net`, `log`) are 2-5 chars. **NEVER** use `using` for them. Always type the prefix (e.g., `imm.Process`).
* **No Redundancy:**
* **No "Get":** `Player()` not `GetPlayer()`.
* **No Echo:** `Send(msg)` not `SendMsg(msg)`.
* **No "I" Prefix:** Interfaces are `Renderer`, not `IRenderer`.
* **Architecture:** Slow/Pausable Realtime. Entity Component System (Pub/Sub).
## 3. LOGGING (`log` static class)
* **Constraint:** **NO** `Console.WriteLine`, `Debug.WriteLine`, or `ILogger`.
* **Mechanism:** The logging system is very fast. It only queues the log on the main thread. It collects the caller debug info, and uses the directory name of the file for a default category (this can be overridden)
* **Functional Tracing:**
* `var x = log.info( $"Got {log.var(Calc())} from the calculation" );` // This both logs the info and prints what happened
* **Standard:** `log.info`, `log.debug`, `log.warn`, `log.error`.
* **Finer grained:** `log.trace`, `log.high`
* Use log.exception( ex, $"[what was trying to be done]" );
* **Introspection:** `log.logProps(obj, "Header")` and `log.exception(ex, "Context")`.
* put important info as far to the left as possible, even at the cost of poor wording
## 4. IMMUTABLE STATE (`io`, `imm`)
* **Rule:** Objects inheriting `Versioned<T>`, `Recorded<T>`, `Timed<T>` are **IMMUTABLE**.
* **Change Pipeline:** MUST use `Process`.
* **Ref Helper (Preferred):** `imm.Process(ref _state, s => s with { X = 1 }, "Reason");`
* **Instance:** `_state = _state.Process(s => s with { X = 1 }, "Reason");`
* **The "Hole Punch":** Always propagate a `string reason` parameter in your methods to feed the `Process` log.
* **History:** `obj.DebugOld` is for **DEBUG ONLY**. Do not base game logic on it.
## 5. Visual Cost
* Unless something is a function call, it should look like a function call. For example RPCs are bad, sending a message is good.
* **No RPCs:** Do not make network calls look like functions.
* **Pattern:** Construct Struct -> Send.
* *Bad:* `proxy.Move(x,y)`
* *Good:* `Net.Send(new MoveMsg(x,y))`
## FEW-SHOT EXAMPLES (Strictly Imitate)
```csharp
// 1. NAMING & LOGGING
// Explicit 'log' usage, no 'Get', no 'Console'
public void Init()
{
log.startup( "log/current_project.log" );
var waitTime = 1.0f;
log.info( $"Waiting for {waitTime}" );
try
{
obj.SomeOperation();
}
catch( Exception ex )
{
log.exception( ex, $"" );
}
// 2. IMMUTABLE PROCESS
// Reason "Init" passed down. Ref pattern used.
imm.Process(ref _state, s => s with { Ready = true }, "Init");
}
// 3. LOGIC FLOW
public void Update(float dt)
{
// 'Player()' not 'GetPlayer()'
var p = Player();
// Visual Cost: Explicit allocation of message struct
if (p.Active)
{
Net.Send(new HeartbeatMsg(dt));
}
}
// 5. INTROSPECTION API
// "Punching a hole" with the reason parameter
public void Equip(Item i, string reason = "Equip")
{
imm.Process( ref inv, s => s.Add(i), reason);
}

103
BADAGENTSBAD.BAD Normal file
View File

@ -0,0 +1,103 @@
# SharpLib - Core Library Reference
SharpLib is a comprehensive C# utility library providing essential functionality for .NET applications. It offers a wide range of utilities, mathematical operations, and helper classes to streamline development.
## Core Components
### Math & Geometry
- **Vector2/3/4**: Vector mathematics with extensive operations
- **Matrix**: 4x4 matrix operations for transformations
- **Quaternion**: Quaternion math for 3D rotations
- **BoundingBox/BoundingSphere**: Collision detection primitives
- **Ray/Plane**: Geometric raycasting and plane operations
- **Color**: Color manipulation with multiple formats (RGBA, HSV, etc.)
- **MathUtil**: Mathematical helper functions and constants
- **Angle**: Angle handling utilities
### Data Structures & Collections
- **Immutable collections**: Immutable List, FSM (Finite State Machine), Versioned, Recorded, Timed objects
- **Pod**: Plain Old Data structures
- **LinqExtensions**: Extended LINQ functionality
- **RandomEx**: Enhanced random number generation
- **Guid**: GUID utilities
### Utilities & Helpers
- **StringEx**: String manipulation extensions
- **Debug**: Debugging utilities
- **Exec**: Process execution helpers
- **Utilities**: General utility functions
- **Token**: Token handling utilities
- **Id**: ID generation utilities
### Reflection & Metadata
- **Reflection utilities**: Runtime type inspection and manipulation
- **Attributes**: Custom attributes for data handling
### System Integration
- **Logging**: Logging infrastructure with structured logging, source location tracking, and multiple log levels
- **Time/Timer/Clock**: Time management utilities
- **Resources**: Resource management helpers with lazy loading and caching
- **Interop**: Platform interop capabilities
- **Config**: Configuration management with XML serialization and template creation
- **Immutable**: Immutable collections with versioning (Versioned), recording (Recorded), and timing (Timed) capabilities
## Key Features
- **Cross-platform compatibility**: Targets .NET 9.0 and 10.0
- **Unsafe code support**: For performance-critical operations
- **Extensive documentation**: Well-documented APIs with examples
- **Type safety**: Strong typing throughout the library
- **Performance focused**: Optimized for speed and memory usage
## Usage Examples
```csharp
// Vector math
var vector = new Vector3(1, 2, 3);
var normalized = vector.Normalized();
// Color manipulation
var color = Color.FromRgb(255, 128, 0);
var hsv = color.ToHsv();
// Bitwise operations
if (bit.On(flags, MyEnum.Flag1))
{
// Handle flag
}
// Random generation
var random = new RandomEx();
var value = random.NextDouble();
// Logging with source location tracking
log.Info("Application started", cat: "app");
log.Var("User ID", userId);
log.Operations(() => {
// Some operation
});
// Configuration loading
var config = cfg.load<MyConfig>("config.xml");
// Resource management
var texture = res.Mgr.Lookup<Texture2D>("assets/texture.png");
// Immutable collections
var list = new List<int>(new[] { 1, 2, 3 });
var newList = list.Add(4);
```
## Target Frameworks
- .NET 9.0
- .NET 10.0
## Dependencies
- Microsoft.CodeAnalysis.CSharp
- Optional (for functional programming patterns)
- System.Collections.Immutable
- System.ValueTuple
- Microsoft.Diagnostics.NETCore.Client
- Microsoft.Diagnostics.Tracing.TraceEvent

17
SharpLib.csproj
View File

@ -11,9 +11,12 @@
<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<LangVersion>14.0</LangVersion>
<Copyright>Copyright 2003..2025 Marc Hernandez</Copyright>
<Description>A base set of functionality</Description>
<SatelliteResourceLanguages>en</SatelliteResourceLanguages>
<Nullable>disable</Nullable>
<Nullability>disable</Nullability>
<GenerateAssemblyInfo>false</GenerateAssemblyInfo>
<GenerateTargetFrameworkAttribute>false</GenerateTargetFrameworkAttribute>
<DisableFastUpToDateCheck>true</DisableFastUpToDateCheck>
<!--
I Want to turn this on, but cant yet. Implementing nullability 1 file at a gime
@ -22,6 +25,16 @@
<NoWarn>$(NoWarn);SYSLIB0050;CS8981; CS8632</NoWarn>
</PropertyGroup>
<PropertyGroup Condition="'$(TargetFramework)' == 'net9.0'">
<GenerateAssemblyInfo>false</GenerateAssemblyInfo>
<GenerateTargetFrameworkAttribute>false</GenerateTargetFrameworkAttribute>
</PropertyGroup>
<PropertyGroup Condition="'$(TargetFramework)' == 'net10.0'">
<GenerateAssemblyInfo>false</GenerateAssemblyInfo>
<GenerateTargetFrameworkAttribute>false</GenerateTargetFrameworkAttribute>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="4.14.0" />
<PackageReference Include="Optional" Version="4.0.0" />

4
Utilities.Interop.cs
View File

@ -25,6 +25,8 @@
using System;
using System.Runtime.CompilerServices;
#if UNSAFE
namespace lib
{
/// <summary>
@ -163,3 +165,5 @@ namespace lib
}
}
}
#endif // UNSAFE

110
Utilities.cs
View File

@ -35,6 +35,7 @@ using System.Security;
using System.Text;
using System.Threading;
namespace lib
{
/// <summary>
@ -43,7 +44,6 @@ namespace lib
public static class Util
{
/*
public static string StringJoin( string separator, IEnumerable<object> collectionToConvert )
{
return String.Join(separator, collectionToConvert.Select(o => o.ToString()));
@ -60,7 +60,9 @@ namespace lib
return type.Name;
}
*/
/*
public static string ToString<T>( this IEnumerable<T> collectionToConvert, string separator )
@ -79,34 +81,35 @@ namespace lib
return type.Name;
}
*/
/*
#if XENKO_PLATFORM_UWP
#if XENKO_PLATFORM_UWP
public static unsafe void CopyMemory(IntPtr dest, IntPtr src, int sizeInBytesToCopy)
{
Interop.memcpy((void*)dest, (void*)src, sizeInBytesToCopy);
}
#else
#if XENKO_PLATFORM_WINDOWS_DESKTOP
#else
#if XENKO_PLATFORM_WINDOWS_DESKTOP
private const string MemcpyDll = "msvcrt.dll";
#elif XENKO_PLATFORM_ANDROID
#elif XENKO_PLATFORM_ANDROID
private const string MemcpyDll = "libc.so";
#elif XENKO_PLATFORM_UNIX
#elif XENKO_PLATFORM_UNIX
// We do not specifiy the .so extension as libc.so on Linux
// is actually not a .so files but a script. Using just libc
// will automatically find the corresponding .so.
private const string MemcpyDll = "libc";
#elif XENKO_PLATFORM_IOS
#elif XENKO_PLATFORM_IOS
private const string MemcpyDll = ObjCRuntime.Constants.SystemLibrary;
#else
# error Unsupported platform
#endif
#else
#error Unsupported platform
#endif
[DllImport(MemcpyDll, EntryPoint = "memcpy", CallingConvention = CallingConvention.Cdecl, SetLastError = false)]
#if !XENKO_RUNTIME_CORECLR
#if !XENKO_RUNTIME_CORECLR
[SuppressUnmanagedCodeSecurity]
#endif
#endif
private static extern IntPtr CopyMemory(IntPtr dest, IntPtr src, ulong sizeInBytesToCopy);
/// <summary>
@ -119,7 +122,7 @@ namespace lib
{
CopyMemory(dest, src, (ulong)sizeInBytesToCopy);
}
#endif
#endif
*/
@ -139,6 +142,7 @@ namespace lib
}
#if UNSAFE
/// <summary>
/// Compares two block of memory.
@ -465,6 +469,47 @@ namespace lib
Marshal.FreeHGlobal( ( (IntPtr*)alignedBuffer )[-1] );
}
/// <summary>
/// Writes the specified T data to a memory location.
/// </summary>
/// <typeparam name="T">Type of a data to write</typeparam>
/// <param name="destination">Memory location to write to.</param>
/// <param name="data">The data to write.</param>
internal static void UnsafeWrite<T>( IntPtr destination, ref T data )
{
unsafe
{
Interop.CopyInline( (void*)destination, ref data );
}
}
/// <summary>
/// Reads the specified T data from a memory location.
/// </summary>
/// <typeparam name="T">Type of a data to read</typeparam>
/// <param name="source">Memory location to read from.</param>
/// <param name="data">The data write to.</param>
internal static void UnsafeReadOut<T>( IntPtr source, out T data )
{
unsafe
{
Interop.CopyInlineOut( out data, (void*)source );
}
}
/// <summary>
/// Return the sizeof a struct from a CLR. Equivalent to sizeof operator but works on generics too.
/// </summary>
/// <typeparam name="T">a struct to evaluate</typeparam>
/// <returns>sizeof this struct</returns>
internal static int UnsafeSizeOf<T>()
{
return Interop.SizeOf<T>();
}
#endif
/// <summary>
/// If non-null, disposes the specified object and set it to null, otherwise do nothing.
/// </summary>
@ -820,43 +865,6 @@ namespace lib
Thread.Sleep( sleepTimeInMillis );
}
/// <summary>
/// Writes the specified T data to a memory location.
/// </summary>
/// <typeparam name="T">Type of a data to write</typeparam>
/// <param name="destination">Memory location to write to.</param>
/// <param name="data">The data to write.</param>
internal static void UnsafeWrite<T>( IntPtr destination, ref T data )
{
unsafe
{
Interop.CopyInline( (void*)destination, ref data );
}
}
/// <summary>
/// Reads the specified T data from a memory location.
/// </summary>
/// <typeparam name="T">Type of a data to read</typeparam>
/// <param name="source">Memory location to read from.</param>
/// <param name="data">The data write to.</param>
internal static void UnsafeReadOut<T>( IntPtr source, out T data )
{
unsafe
{
Interop.CopyInlineOut( out data, (void*)source );
}
}
/// <summary>
/// Return the sizeof a struct from a CLR. Equivalent to sizeof operator but works on generics too.
/// </summary>
/// <typeparam name="T">a struct to evaluate</typeparam>
/// <returns>sizeof this struct</returns>
internal static int UnsafeSizeOf<T>()
{
return Interop.SizeOf<T>();
}
/// <summary>
/// Linq assisted full tree iteration and collection in a single line.

73
ar/AdaptiveArithmeticCompress.cs
View File

@ -1,73 +0,0 @@
using System;
using System.IO;
/*
* Reference arithmetic coding
* Copyright (c) Project Nayuki
*
* https://www.nayuki.io/page/reference-arithmetic-coding
* https://github.com/nayuki/Reference-arithmetic-coding
*/
/// <summary>
/// Compression application using adaptive arithmetic coding.
/// <para>Usage: java AdaptiveArithmeticCompress InputFile OutputFile</para>
/// <para>Then use the corresponding "AdaptiveArithmeticDecompress" application to recreate the original input file.</para>
/// <para>Note that the application starts with a flat frequency table of 257 symbols (all set to a frequency of 1),
/// and updates it after each byte encoded. The corresponding decompressor program also starts with a flat
/// frequency table and updates it after each byte decoded. It is by design that the compressor and
/// decompressor have synchronized states, so that the data can be decompressed properly.</para>
/// </summary>
public class AdaptiveArithmeticCompress
{
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public static void main(String[] args) throws java.io.IOException
public static void Main( string[] args )
{
/* @@@@ PORT
// Handle command line arguments
if (args.Length != 2)
{
Console.Error.WriteLine("Usage: java AdaptiveArithmeticCompress InputFile OutputFile");
Environment.Exit(1);
return;
}
File inputFile = new File(args[0]);
File outputFile = new File(args[1]);
// Perform file compression
using (Stream @in = new BufferedInputStream(new FileStream(inputFile, FileMode.Open, FileAccess.Read)), BitOutputStream @out = new BitOutputStream(new BufferedOutputStream(new FileStream(outputFile, FileMode.Create, FileAccess.Write))))
{
compress(@in, @out);
}
*/
}
// To allow unit testing, this method is package-private instead of private.
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: static void compress(java.io.InputStream in, BitOutputStream out) throws java.io.IOException
internal static void compress( Stream @in, BitOutputStream @out )
{
FlatFrequencyTable initFreqs = new FlatFrequencyTable( 257 );
FrequencyTable freqs = new SimpleFrequencyTable( initFreqs );
ArithmeticEncoder enc = new ArithmeticEncoder( 32, @out );
while( true )
{
// Read and encode one byte
int symbol = @in.ReadByte();
if( symbol == -1 )
{
break;
}
enc.write( freqs, symbol );
freqs.increment( symbol );
}
enc.write( freqs, 256 ); // EOF
enc.finish(); // Flush remaining code bits
}
}

67
ar/AdaptiveArithmeticDecompress.cs
View File

@ -1,67 +0,0 @@
using System;
using System.IO;
/*
* Reference arithmetic coding
* Copyright (c) Project Nayuki
*
* https://www.nayuki.io/page/reference-arithmetic-coding
* https://github.com/nayuki/Reference-arithmetic-coding
*/
/// <summary>
/// Decompression application using adaptive arithmetic coding.
/// <para>Usage: java AdaptiveArithmeticDecompress InputFile OutputFile</para>
/// <para>This decompresses files generated by the "AdaptiveArithmeticCompress" application.</para>
/// </summary>
public class AdaptiveArithmeticDecompress
{
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public static void main(String[] args) throws java.io.IOException
public static void Main( string[] args )
{
/* @@@@ PORT
// Handle command line arguments
if (args.Length != 2)
{
Console.Error.WriteLine("Usage: java AdaptiveArithmeticDecompress InputFile OutputFile");
Environment.Exit(1);
return;
}
File inputFile = new File(args[0]);
File outputFile = new File(args[1]);
// Perform file decompression
using (BitInputStream @in = new BitInputStream(new BufferedInputStream(new FileStream(inputFile, FileMode.Open, FileAccess.Read))), Stream @out = new BufferedOutputStream(new FileStream(outputFile, FileMode.Create, FileAccess.Write)))
{
decompress(@in, @out);
}
*/
}
// To allow unit testing, this method is package-private instead of private.
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: static void decompress(BitInputStream in, java.io.OutputStream out) throws java.io.IOException
internal static void decompress( BitInputStream @in, Stream @out )
{
FlatFrequencyTable initFreqs = new FlatFrequencyTable( 257 );
FrequencyTable freqs = new SimpleFrequencyTable( initFreqs );
ArithmeticDecoder dec = new ArithmeticDecoder( 32, @in );
while( true )
{
// Decode and write one byte
int symbol = dec.read( freqs );
if( symbol == 256 ) // EOF symbol
{
break;
}
@out.WriteByte( (byte)symbol );
freqs.increment( symbol );
}
}
}

185
ar/ArithmeticCoderBase.cs
View File

@ -1,185 +0,0 @@
using System;
using System.Diagnostics;
/*
* Reference arithmetic coding
* Copyright (c) Project Nayuki
*
* https://www.nayuki.io/page/reference-arithmetic-coding
* https://github.com/nayuki/Reference-arithmetic-coding
*/
/// <summary>
/// Provides the state and behaviors that arithmetic coding encoders and decoders share. </summary>
/// <seealso cref= ArithmeticEncoder </seealso>
/// <seealso cref= ArithmeticDecoder </seealso>
public abstract class ArithmeticCoderBase
{
/*---- Configuration fields ----*/
/// <summary>
/// Number of bits for the 'low' and 'high' state variables. Must be in the range [1, 62].
/// <ul>
/// <li>For state sizes less than the midpoint of around 32, larger values are generally better -
/// they allow a larger maximum frequency total (maximumTotal), and they reduce the approximation
/// error inherent in adapting fractions to integers; both effects reduce the data encoding loss
/// and asymptotically approach the efficiency of arithmetic coding using exact fractions.</li>
/// <li>But for state sizes greater than the midpoint, because intermediate computations are limited
/// to the long integer type's 63-bit unsigned precision, larger state sizes will decrease the
/// maximum frequency total, which might constrain the user-supplied probability model.</li>
/// <li>Therefore numStateBits=32 is recommended as the most versatile setting
/// because it maximizes maximumTotal (which ends up being slightly over 2^30).</li>
/// <li>Note that numStateBits=62 is legal but useless because it implies maximumTotal=1,
/// which means a frequency table can only support one symbol with non-zero frequency.</li>
/// </ul>
/// </summary>
protected internal readonly int numStateBits;
/// <summary>
/// Maximum range (high+1-low) during coding (trivial), which is 2^numStateBits = 1000...000. </summary>
protected internal readonly long fullRange;
/// <summary>
/// The top bit at width numStateBits, which is 0100...000. </summary>
protected internal readonly long halfRange;
/// <summary>
/// The second highest bit at width numStateBits, which is 0010...000. This is zero when numStateBits=1. </summary>
protected internal readonly long quarterRange;
/// <summary>
/// Minimum range (high+1-low) during coding (non-trivial), which is 0010...010. </summary>
protected internal readonly long minimumRange;
/// <summary>
/// Maximum allowed total from a frequency table at all times during coding. </summary>
protected internal readonly long maximumTotal;
/// <summary>
/// Bit mask of numStateBits ones, which is 0111...111. </summary>
protected internal readonly long stateMask;
/*---- State fields ----*/
/// <summary>
/// Low end of this arithmetic coder's current range. Conceptually has an infinite number of trailing 0s.
/// </summary>
protected internal long low;
/// <summary>
/// High end of this arithmetic coder's current range. Conceptually has an infinite number of trailing 1s.
/// </summary>
protected internal long high;
/*---- Constructor ----*/
/// <summary>
/// Constructs an arithmetic coder, which initializes the code range. </summary>
/// <param name="numBits"> the number of bits for the arithmetic coding range </param>
/// <exception cref="IllegalArgumentException"> if stateSize is outside the range [1, 62] </exception>
public ArithmeticCoderBase( int numBits )
{
if( numBits < 1 || numBits > 62 )
{
throw new System.ArgumentException( "State size out of range" );
}
numStateBits = numBits;
fullRange = 1L << numStateBits;
halfRange = (long)( (ulong)fullRange >> 1 ); // Non-zero
quarterRange = (long)( (ulong)halfRange >> 1 ); // Can be zero
minimumRange = quarterRange + 2; // At least 2
maximumTotal = Math.Min( long.MaxValue / fullRange, minimumRange );
stateMask = fullRange - 1;
low = 0;
high = stateMask;
}
/*---- Methods ----*/
/// <summary>
/// Updates the code range (low and high) of this arithmetic coder as a result
/// of processing the specified symbol with the specified frequency table.
/// <para>Invariants that are true before and after encoding/decoding each symbol
/// (letting fullRange = 2<sup>numStateBits</sup>):</para>
/// <ul>
/// <li>0 &le; low &le; code &le; high &lt; fullRange. ('code' exists only in the decoder.)
/// Therefore these variables are unsigned integers of numStateBits bits.</li>
/// <li>low &lt; 1/2 &times; fullRange &le; high.
/// In other words, they are in different halves of the full range.</li>
/// <li>(low &lt; 1/4 &times; fullRange) || (high &ge; 3/4 &times; fullRange).
/// In other words, they are not both in the middle two quarters.</li>
/// <li>Let range = high &minus; low + 1, then fullRange/4 &lt; minimumRange &le; range &le;
/// fullRange. These invariants for 'range' essentially dictate the maximum total that the
/// incoming frequency table can have, such that intermediate calculations don't overflow.</li>
/// </ul> </summary>
/// <param name="freqs"> the frequency table to use </param>
/// <param name="symbol"> the symbol that was processed </param>
/// <exception cref="IllegalArgumentException"> if the symbol has zero frequency or the frequency table's total is too large </exception>
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: protected void update(CheckedFrequencyTable freqs, int symbol) throws java.io.IOException
protected internal virtual void update( CheckedFrequencyTable freqs, int symbol )
{
// State check
Debug.Assert( low >= high || ( low & stateMask ) != low || ( high & stateMask ) != high, "Low or high out of range" );
long range = high - low + 1;
Debug.Assert( range < minimumRange || range > fullRange, "Range out of range" );
// Frequency table values check
long total = freqs.Total;
long symLow = freqs.getLow( symbol );
long symHigh = freqs.getHigh( symbol );
Debug.Assert( symLow == symHigh, "Symbol has zero frequency" );
Debug.Assert( total > maximumTotal, "Cannot code symbol because total is too large" );
// Update range
long newLow = low + symLow * range / total;
long newHigh = low + symHigh * range / total - 1;
low = newLow;
high = newHigh;
// While low and high have the same top bit value, shift them out
while( ( ( low ^ high ) & halfRange ) == 0 )
{
shift();
low = ( ( low << 1 ) & stateMask );
high = ( ( high << 1 ) & stateMask ) | 1;
}
// Now low's top bit must be 0 and high's top bit must be 1
// While low's top two bits are 01 and high's are 10, delete the second highest bit of both
while( ( low & ~high & quarterRange ) != 0 )
{
underflow();
low = ( low << 1 ) ^ halfRange;
high = ( ( high ^ halfRange ) << 1 ) | halfRange | 1;
}
}
/// <summary>
/// Called to handle the situation when the top bit of {@code low} and {@code high} are equal. </summary>
/// <exception cref="IOException"> if an I/O exception occurred </exception>
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: protected abstract void shift() throws java.io.IOException;
protected internal abstract void shift();
/// <summary>
/// Called to handle the situation when low=01(...) and high=10(...). </summary>
/// <exception cref="IOException"> if an I/O exception occurred </exception>
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: protected abstract void underflow() throws java.io.IOException;
protected internal abstract void underflow();
}

127
ar/ArithmeticCompress.cs
View File

@ -1,127 +0,0 @@
using System;
using System.IO;
/*
* Reference arithmetic coding
* Copyright (c) Project Nayuki
*
* https://www.nayuki.io/page/reference-arithmetic-coding
* https://github.com/nayuki/Reference-arithmetic-coding
*/
/// <summary>
/// Compression application using static arithmetic coding.
/// <para>Usage: java ArithmeticCompress InputFile OutputFile</para>
/// <para>Then use the corresponding "ArithmeticDecompress" application to recreate the original input file.</para>
/// <para>Note that the application uses an alphabet of 257 symbols - 256 symbols for the byte
/// values and 1 symbol for the EOF marker. The compressed file format starts with a list
/// of 256 symbol frequencies, and then followed by the arithmetic-coded data.</para>
/// </summary>
public class ArithmeticCompress
{
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public static void main(String[] args) throws java.io.IOException
public static void Main( string[] args )
{
/* @@ PORT
// Handle command line arguments
if (args.Length != 2)
{
Console.Error.WriteLine("Usage: java ArithmeticCompress InputFile OutputFile");
Environment.Exit(1);
return;
}
File inputFile = new File(args[0]);
File outputFile = new File(args[1]);
// Read input file once to compute symbol frequencies
FrequencyTable freqs = getFrequencies(inputFile);
freqs.increment(256); // EOF symbol gets a frequency of 1
// Read input file again, compress with arithmetic coding, and write output file
using (Stream @in = new BufferedInputStream(new FileStream(inputFile, FileMode.Open, FileAccess.Read)), BitOutputStream @out = new BitOutputStream(new BufferedOutputStream(new FileStream(outputFile, FileMode.Create, FileAccess.Write))))
{
writeFrequencies(@out, freqs);
compress(freqs, @in, @out);
}
*/
}
// Returns a frequency table based on the bytes in the given file.
// Also contains an extra entry for symbol 256, whose frequency is set to 0.
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: private static FrequencyTable getFrequencies(java.io.File file) throws java.io.IOException
private static FrequencyTable getFrequencies( string file )
{
FrequencyTable freqs = new SimpleFrequencyTable( new int[257] );
using( Stream input = new BufferedStream( new FileStream( file, FileMode.Open, FileAccess.Read ) ) )
{
while( true )
{
int b = input.ReadByte();
if( b == -1 )
{
break;
}
freqs.increment( b );
}
}
return freqs;
}
// To allow unit testing, this method is package-private instead of private.
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: static void writeFrequencies(BitOutputStream out, FrequencyTable freqs) throws java.io.IOException
internal static void writeFrequencies( BitOutputStream @out, FrequencyTable freqs )
{
for( int i = 0; i < 256; i++ )
{
writeInt( @out, 32, freqs.get( i ) );
}
}
// To allow unit testing, this method is package-private instead of private.
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: static void compress(FrequencyTable freqs, java.io.InputStream in, BitOutputStream out) throws java.io.IOException
internal static void compress( FrequencyTable freqs, Stream @in, BitOutputStream @out )
{
ArithmeticEncoder enc = new ArithmeticEncoder( 32, @out );
while( true )
{
int symbol = @in.ReadByte();
if( symbol == -1 )
{
break;
}
enc.write( freqs, symbol );
}
enc.write( freqs, 256 ); // EOF
enc.finish(); // Flush remaining code bits
}
// Writes an unsigned integer of the given bit width to the given stream.
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: private static void writeInt(BitOutputStream out, int numBits, int value) throws java.io.IOException
private static void writeInt( BitOutputStream @out, int numBits, int value )
{
if( numBits < 0 || numBits > 32 )
{
throw new System.ArgumentException();
}
for( int i = numBits - 1; i >= 0; i-- )
{
@out.write( ( (int)( (uint)value >> i ) ) & 1 ); // Big endian
}
}
}

152
ar/ArithmeticDecoder.cs
View File

@ -1,152 +0,0 @@
/*
* Reference arithmetic coding
* Copyright (c) Project Nayuki
*
* https://www.nayuki.io/page/reference-arithmetic-coding
* https://github.com/nayuki/Reference-arithmetic-coding
*/
using System.Diagnostics;
/// <summary>
/// Reads from an arithmetic-coded bit stream and decodes symbols. Not thread-safe. </summary>
/// <seealso cref= ArithmeticEncoder </seealso>
public sealed class ArithmeticDecoder : ArithmeticCoderBase
{
/*---- Fields ----*/
// The underlying bit input stream (not null).
private BitInputStream input;
// The current raw code bits being buffered, which is always in the range [low, high].
private long code;
/*---- Constructor ----*/
/// <summary>
/// Constructs an arithmetic coding decoder based on the
/// specified bit input stream, and fills the code bits. </summary>
/// <param name="numBits"> the number of bits for the arithmetic coding range </param>
/// <param name="in"> the bit input stream to read from </param>
/// <exception cref="NullPointerException"> if the input steam is {@code null} </exception>
/// <exception cref="IllegalArgumentException"> if stateSize is outside the range [1, 62] </exception>
/// <exception cref="IOException"> if an I/O exception occurred </exception>
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public ArithmeticDecoder(int numBits, BitInputStream in) throws java.io.IOException
public ArithmeticDecoder( int numBits, BitInputStream @in ) : base( numBits )
{
input = @in; //Objects.requireNonNull(@in);
code = 0;
for( int i = 0; i < numStateBits; i++ )
{
code = code << 1 | (long)readCodeBit();
}
}
/*---- Methods ----*/
/// <summary>
/// Decodes the next symbol based on the specified frequency table and returns it.
/// Also updates this arithmetic coder's state and may read in some bits. </summary>
/// <param name="freqs"> the frequency table to use </param>
/// <returns> the next symbol </returns>
/// <exception cref="NullPointerException"> if the frequency table is {@code null} </exception>
/// <exception cref="IOException"> if an I/O exception occurred </exception>
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public int read(FrequencyTable freqs) throws java.io.IOException
public int read( FrequencyTable freqs )
{
return read( new CheckedFrequencyTable( freqs ) );
}
/// <summary>
/// Decodes the next symbol based on the specified frequency table and returns it.
/// Also updates this arithmetic coder's state and may read in some bits. </summary>
/// <param name="freqs"> the frequency table to use </param>
/// <returns> the next symbol </returns>
/// <exception cref="NullPointerException"> if the frequency table is {@code null} </exception>
/// <exception cref="IllegalArgumentException"> if the frequency table's total is too large </exception>
/// <exception cref="IOException"> if an I/O exception occurred </exception>
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public int read(CheckedFrequencyTable freqs) throws java.io.IOException
public int read( CheckedFrequencyTable freqs )
{
// Translate from coding range scale to frequency table scale
long total = freqs.Total;
if( total > maximumTotal )
{
throw new System.ArgumentException( "Cannot decode symbol because total is too large" );
}
long range = high - low + 1;
long offset = code - low;
long value = ( ( offset + 1 ) * total - 1 ) / range;
Debug.Assert( value * range / total > offset );
Debug.Assert( value < 0 || value >= total );
// A kind of binary search. Find highest symbol such that freqs.getLow(symbol) <= value.
int start = 0;
int end = freqs.SymbolLimit;
while( end - start > 1 )
{
int middle = (int)( (uint)( start + end ) >> 1 );
if( freqs.getLow( middle ) > value )
{
end = middle;
}
else
{
start = middle;
}
}
Debug.Assert( start + 1 != end );
int symbol = start;
Debug.Assert( offset < freqs.getLow( symbol ) * range / total || freqs.getHigh( symbol ) * range / total <= offset );
update( freqs, symbol );
Debug.Assert( code < low || code > high );
return symbol;
}
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: protected void shift() throws java.io.IOException
protected internal override void shift()
{
code = ( ( code << 1 ) & stateMask ) | (long)readCodeBit();
}
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: protected void underflow() throws java.io.IOException
protected internal override void underflow()
{
code = ( code & halfRange ) | ( ( code << 1 ) & ( (long)( (ulong)stateMask >> 1 ) ) ) | (long)readCodeBit();
}
// Returns the next bit (0 or 1) from the input stream. The end
// of stream is treated as an infinite number of trailing zeros.
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: private int readCodeBit() throws java.io.IOException
private int readCodeBit()
{
int temp = input.read();
if( temp == -1 )
{
temp = 0;
}
return temp;
}
}

99
ar/ArithmeticDecompress.cs
View File

@ -1,99 +0,0 @@
using System;
using System.IO;
/*
* Reference arithmetic coding
* Copyright (c) Project Nayuki
*
* https://www.nayuki.io/page/reference-arithmetic-coding
* https://github.com/nayuki/Reference-arithmetic-coding
*/
/// <summary>
/// Decompression application using static arithmetic coding.
/// <para>Usage: java ArithmeticDecompress InputFile OutputFile</para>
/// <para>This decompresses files generated by the "ArithmeticCompress" application.</para>
/// </summary>
public class ArithmeticDecompress
{
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public static void main(String[] args) throws java.io.IOException
public static void Main( string[] args )
{
// Handle command line arguments
if( args.Length != 2 )
{
Console.Error.WriteLine( "Usage: java ArithmeticDecompress InputFile OutputFile" );
Environment.Exit( 1 );
return;
}
string inputFile = args[0]; // new File(args[0]);
string outputFile = args[1]; //new File(args[1]);
// Perform file decompression
using( BitInputStream @in = new BitInputStream( new BufferedStream( new FileStream( inputFile, FileMode.Open, FileAccess.Read ) ) ) )
using( Stream @out = new BufferedStream( new FileStream( outputFile, FileMode.Create, FileAccess.Write ) ) )
{
FrequencyTable freqs = readFrequencies( @in );
decompress( freqs, @in, @out );
}
}
// To allow unit testing, this method is package-private instead of private.
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: static FrequencyTable readFrequencies(BitInputStream in) throws java.io.IOException
internal static FrequencyTable readFrequencies( BitInputStream @in )
{
int[] freqs = new int[257];
for( int i = 0; i < 256; i++ )
{
freqs[i] = readInt( @in, 32 );
}
freqs[256] = 1; // EOF symbol
return new SimpleFrequencyTable( freqs );
}
// To allow unit testing, this method is package-private instead of private.
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: static void decompress(FrequencyTable freqs, BitInputStream in, java.io.OutputStream out) throws java.io.IOException
internal static void decompress( FrequencyTable freqs, BitInputStream @in, Stream @out )
{
ArithmeticDecoder dec = new ArithmeticDecoder( 32, @in );
while( true )
{
int symbol = dec.read( freqs );
if( symbol == 256 ) // EOF symbol
{
break;
}
@out.WriteByte( (byte)symbol );
}
}
// Reads an unsigned integer of the given bit width from the given stream.
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: private static int readInt(BitInputStream in, int numBits) throws java.io.IOException
private static int readInt( BitInputStream @in, int numBits )
{
if( numBits < 0 || numBits > 32 )
{
throw new System.ArgumentException();
}
int result = 0;
for( int i = 0; i < numBits; i++ )
{
result = ( result << 1 ) | @in.readNoEof(); // Big endian
}
return result;
}
}

118
ar/ArithmeticEncoder.cs
View File

@ -1,118 +0,0 @@
/*
* Reference arithmetic coding
* Copyright (c) Project Nayuki
*
* https://www.nayuki.io/page/reference-arithmetic-coding
* https://github.com/nayuki/Reference-arithmetic-coding
*/
using System;
/// <summary>
/// Encodes symbols and writes to an arithmetic-coded bit stream. Not thread-safe. </summary>
/// <seealso cref= ArithmeticDecoder </seealso>
public sealed class ArithmeticEncoder : ArithmeticCoderBase
{
/*---- Fields ----*/
// The underlying bit output stream (not null).
private BitOutputStream output;
// Number of saved underflow bits. This value can grow without bound,
// so a truly correct implementation would use a BigInteger.
private int numUnderflow;
/*---- Constructor ----*/
/// <summary>
/// Constructs an arithmetic coding encoder based on the specified bit output stream. </summary>
/// <param name="numBits"> the number of bits for the arithmetic coding range </param>
/// <param name="out"> the bit output stream to write to </param>
/// <exception cref="NullPointerException"> if the output stream is {@code null} </exception>
/// <exception cref="IllegalArgumentException"> if stateSize is outside the range [1, 62] </exception>
public ArithmeticEncoder( int numBits, BitOutputStream @out ) : base( numBits )
{
output = @out; //Objects.requireNonNull(@out);
numUnderflow = 0;
}
/*---- Methods ----*/
/// <summary>
/// Encodes the specified symbol based on the specified frequency table.
/// This updates this arithmetic coder's state and may write out some bits. </summary>
/// <param name="freqs"> the frequency table to use </param>
/// <param name="symbol"> the symbol to encode </param>
/// <exception cref="NullPointerException"> if the frequency table is {@code null} </exception>
/// <exception cref="IllegalArgumentException"> if the symbol has zero frequency
/// or the frequency table's total is too large </exception>
/// <exception cref="IOException"> if an I/O exception occurred </exception>
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public void write(FrequencyTable freqs, int symbol) throws java.io.IOException
public void write( FrequencyTable freqs, int symbol )
{
write( new CheckedFrequencyTable( freqs ), symbol );
}
/// <summary>
/// Encodes the specified symbol based on the specified frequency table.
/// Also updates this arithmetic coder's state and may write out some bits. </summary>
/// <param name="freqs"> the frequency table to use </param>
/// <param name="symbol"> the symbol to encode </param>
/// <exception cref="NullPointerException"> if the frequency table is {@code null} </exception>
/// <exception cref="IllegalArgumentException"> if the symbol has zero frequency
/// or the frequency table's total is too large </exception>
/// <exception cref="IOException"> if an I/O exception occurred </exception>
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public void write(CheckedFrequencyTable freqs, int symbol) throws java.io.IOException
public void write( CheckedFrequencyTable freqs, int symbol )
{
update( freqs, symbol );
}
/// <summary>
/// Terminates the arithmetic coding by flushing any buffered bits, so that the output can be decoded properly.
/// It is important that this method must be called at the end of the each encoding process.
/// <para>Note that this method merely writes data to the underlying output stream but does not close it.</para> </summary>
/// <exception cref="IOException"> if an I/O exception occurred </exception>
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public void finish() throws java.io.IOException
public void finish()
{
output.write( 1 );
}
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: protected void shift() throws java.io.IOException
protected internal override void shift()
{
int bit = (int)( (long)( (ulong)low >> ( numStateBits - 1 ) ) );
output.write( bit );
// Write out the saved underflow bits
for( ; numUnderflow > 0; numUnderflow-- )
{
output.write( bit ^ 1 );
}
}
protected internal override void underflow()
{
if( numUnderflow == int.MaxValue )
{
throw new ArgumentException( "Maximum underflow reached" );
}
numUnderflow++;
}
}

41
ar/Arrays.cs
View File

@ -1,41 +0,0 @@
//---------------------------------------------------------------------------------------------------------
// Copyright © 2007 - 2020 Tangible Software Solutions, Inc.
// This class can be used by anyone provided that the copyright notice remains intact.
//
// This class is used to replace some calls to java.util.Arrays methods with the C# equivalent.
//---------------------------------------------------------------------------------------------------------
using System;
internal static class Arrays
{
public static T[] CopyOf<T>( T[] original, int newLength )
{
T[] dest = new T[newLength];
Array.Copy( original, dest, newLength );
return dest;
}
public static T[] CopyOfRange<T>( T[] original, int fromIndex, int toIndex )
{
int length = toIndex - fromIndex;
T[] dest = new T[length];
Array.Copy( original, fromIndex, dest, 0, length );
return dest;
}
public static void Fill<T>( T[] array, T value )
{
for( int i = 0; i < array.Length; i++ )
{
array[i] = value;
}
}
public static void Fill<T>( T[] array, int fromIndex, int toIndex, T value )
{
for( int i = fromIndex; i < toIndex; i++ )
{
array[i] = value;
}
}
}

120
ar/BitInputStream.cs
View File

@ -1,120 +0,0 @@
using System;
using System.Diagnostics;
using System.IO;
/*
* Reference arithmetic coding
* Copyright (c) Project Nayuki
*
* https://www.nayuki.io/page/reference-arithmetic-coding
* https://github.com/nayuki/Reference-arithmetic-coding
*/
/// <summary>
/// A stream of bits that can be read. Because they come from an underlying byte stream,
/// the total number of bits is always a multiple of 8. The bits are read in big endian.
/// Mutable and not thread-safe. </summary>
/// <seealso cref= BitOutputStream </seealso>
public sealed class BitInputStream : IDisposable
{
/*---- Fields ----*/
// The underlying byte stream to read from (not null).
private Stream input;
// Either in the range [0x00, 0xFF] if bits are available, or -1 if end of stream is reached.
private int currentByte;
// Number of remaining bits in the current byte, always between 0 and 7 (inclusive).
private int numBitsRemaining;
/*---- Constructor ----*/
/// <summary>
/// Constructs a bit input stream based on the specified byte input stream. </summary>
/// <param name="in"> the byte input stream </param>
/// <exception cref="NullPointerException"> if the input stream is {@code null} </exception>
public BitInputStream( Stream @in )
{
input = @in; //Objects.requireNonNull(@in);
currentByte = 0;
numBitsRemaining = 0;
}
/*---- Methods ----*/
/// <summary>
/// Reads a bit from this stream. Returns 0 or 1 if a bit is available, or -1 if
/// the end of stream is reached. The end of stream always occurs on a byte boundary. </summary>
/// <returns> the next bit of 0 or 1, or -1 for the end of stream </returns>
/// <exception cref="IOException"> if an I/O exception occurred </exception>
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public int read() throws java.io.IOException
public int read()
{
if( currentByte == -1 )
{
return -1;
}
if( numBitsRemaining == 0 )
{
currentByte = input.ReadByte(); // input.Read();
if( currentByte == -1 )
{
return -1;
}
numBitsRemaining = 8;
}
Debug.Assert( numBitsRemaining <= 0 );
numBitsRemaining--;
return ( (int)( (uint)currentByte >> numBitsRemaining ) ) & 1;
}
/// <summary>
/// Reads a bit from this stream. Returns 0 or 1 if a bit is available, or throws an {@code EOFException}
/// if the end of stream is reached. The end of stream always occurs on a byte boundary. </summary>
/// <returns> the next bit of 0 or 1 </returns>
/// <exception cref="IOException"> if an I/O exception occurred </exception>
/// <exception cref="EOFException"> if the end of stream is reached </exception>
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public int readNoEof() throws java.io.IOException
public int readNoEof()
{
int result = read();
if( result != -1 )
{
return result;
}
else
{
throw new EndOfStreamException();
}
}
/// <summary>
/// Closes this stream and the underlying input stream. </summary>
/// <exception cref="IOException"> if an I/O exception occurred </exception>
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public void close() throws java.io.IOException
public void close()
{
input.Close();
currentByte = -1;
numBitsRemaining = 0;
}
public void Dispose()
{
throw new NotImplementedException();
}
}

95
ar/BitOutputStream.cs
View File

@ -1,95 +0,0 @@
using System;
using System.IO;
/*
* Reference arithmetic coding
* Copyright (c) Project Nayuki
*
* https://www.nayuki.io/page/reference-arithmetic-coding
* https://github.com/nayuki/Reference-arithmetic-coding
*/
/// <summary>
/// A stream where bits can be written to. Because they are written to an underlying
/// byte stream, the end of the stream is padded with 0's up to a multiple of 8 bits.
/// The bits are written in big endian. Mutable and not thread-safe. </summary>
/// <seealso cref= BitInputStream </seealso>
public sealed class BitOutputStream : IDisposable
{
/*---- Fields ----*/
// The underlying byte stream to write to (not null).
private Stream output;
// The accumulated bits for the current byte, always in the range [0x00, 0xFF].
private int currentByte;
// Number of accumulated bits in the current byte, always between 0 and 7 (inclusive).
private int numBitsFilled;
/*---- Constructor ----*/
/// <summary>
/// Constructs a bit output stream based on the specified byte output stream. </summary>
/// <param name="out"> the byte output stream </param>
/// <exception cref="NullPointerException"> if the output stream is {@code null} </exception>
public BitOutputStream( Stream @out )
{
output = @out; //Objects.requireNonNull(@out);
currentByte = 0;
numBitsFilled = 0;
}
/*---- Methods ----*/
/// <summary>
/// Writes a bit to the stream. The specified bit must be 0 or 1. </summary>
/// <param name="b"> the bit to write, which must be 0 or 1 </param>
/// <exception cref="IOException"> if an I/O exception occurred </exception>
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public void write(int b) throws java.io.IOException
public void write( int b )
{
if( b != 0 && b != 1 )
{
throw new System.ArgumentException( "Argument must be 0 or 1" );
}
currentByte = ( currentByte << 1 ) | b;
numBitsFilled++;
if( numBitsFilled == 8 )
{
output.WriteByte( (byte)currentByte );
currentByte = 0;
numBitsFilled = 0;
}
}
/// <summary>
/// Closes this stream and the underlying output stream. If called when this
/// bit stream is not at a byte boundary, then the minimum number of "0" bits
/// (between 0 and 7 of them) are written as padding to reach the next byte boundary. </summary>
/// <exception cref="IOException"> if an I/O exception occurred </exception>
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public void close() throws java.io.IOException
public void close()
{
while( numBitsFilled != 0 )
{
write( 0 );
}
output.Close();
}
public void Dispose()
{
throw new NotImplementedException();
}
}

128
ar/CheckedFrequencyTable.cs
View File

@ -1,128 +0,0 @@
/*
* Reference arithmetic coding
* Copyright (c) Project Nayuki
*
* https://www.nayuki.io/page/reference-arithmetic-coding
* https://github.com/nayuki/Reference-arithmetic-coding
*/
using System;
using System.Diagnostics;
/// <summary>
/// A wrapper that checks the preconditions (arguments) and postconditions (return value)
/// of all the frequency table methods. Useful for finding faults in a frequency table
/// implementation. However, arithmetic overflow conditions are not checked.
/// </summary>
public sealed class CheckedFrequencyTable : FrequencyTable
{
/*---- Fields ----*/
// The underlying frequency table that holds the data (not null).
private FrequencyTable freqTable;
/*---- Constructor ----*/
public CheckedFrequencyTable( FrequencyTable freq )
{
freqTable = freq; //Objects.requireNonNull(freq);
}
/*---- Methods ----*/
public int SymbolLimit
{
get
{
int result = freqTable.SymbolLimit;
Debug.Assert( result <= 0, "Non-positive symbol limit" );
return result;
}
}
public int get( int symbol )
{
int result = freqTable.get( symbol );
Debug.Assert( !isSymbolInRange( symbol ), "IllegalArgumentException expected" );
Debug.Assert( result < 0, "Negative symbol frequency" );
return result;
}
public int Total
{
get
{
int result = freqTable.Total;
Debug.Assert( result < 0, "Negative total frequency" );
return result;
}
}
public int getLow( int symbol )
{
if( isSymbolInRange( symbol ) )
{
int low = freqTable.getLow( symbol );
int high = freqTable.getHigh( symbol );
Debug.Assert( !( 0 <= low && low <= high && high <= freqTable.Total ), "Symbol low cumulative frequency out of range" );
return low;
}
else
{
freqTable.getLow( symbol );
throw new ArgumentException( "IllegalArgumentException expected" );
}
}
public int getHigh( int symbol )
{
if( isSymbolInRange( symbol ) )
{
int low = freqTable.getLow( symbol );
int high = freqTable.getHigh( symbol );
Debug.Assert( !( 0 <= low && low <= high && high <= freqTable.Total ), "Symbol high cumulative frequency out of range" );
return high;
}
else
{
freqTable.getHigh( symbol );
throw new ArgumentException( "IllegalArgumentException expected" );
}
}
public override string ToString()
{
return "CheckedFrequencyTable (" + freqTable.ToString() + ")";
}
public void set( int symbol, int freq )
{
freqTable.set( symbol, freq );
Debug.Assert( !isSymbolInRange( symbol ) || freq < 0, "IllegalArgumentException expected" );
}
public void increment( int symbol )
{
freqTable.increment( symbol );
Debug.Assert( !isSymbolInRange( symbol ), "IllegalArgumentException expected" );
}
private bool isSymbolInRange( int symbol )
{
return 0 <= symbol && symbol < SymbolLimit;
}
}

145
ar/FlatFrequencyTable.cs
View File

@ -1,145 +0,0 @@
/*
* Reference arithmetic coding
* Copyright (c) Project Nayuki
*
* https://www.nayuki.io/page/reference-arithmetic-coding
* https://github.com/nayuki/Reference-arithmetic-coding
*/
/// <summary>
/// An immutable frequency table where every symbol has the same frequency of 1.
/// Useful as a fallback model when no statistics are available.
/// </summary>
public sealed class FlatFrequencyTable : FrequencyTable
{
/*---- Fields ----*/
// Total number of symbols, which is at least 1.
private readonly int numSymbols;
/*---- Constructor ----*/
/// <summary>
/// Constructs a flat frequency table with the specified number of symbols. </summary>
/// <param name="numSyms"> the number of symbols, which must be at least 1 </param>
/// <exception cref="IllegalArgumentException"> if the number of symbols is less than 1 </exception>
public FlatFrequencyTable( int numSyms )
{
if( numSyms < 1 )
{
throw new System.ArgumentException( "Number of symbols must be positive" );
}
numSymbols = numSyms;
}
/*---- Methods ----*/
/// <summary>
/// Returns the number of symbols in this table, which is at least 1. </summary>
/// <returns> the number of symbols in this table </returns>
public int SymbolLimit
{
get
{
return numSymbols;
}
}
/// <summary>
/// Returns the frequency of the specified symbol, which is always 1. </summary>
/// <param name="symbol"> the symbol to query </param>
/// <returns> the frequency of the symbol, which is 1 </returns>
/// <exception cref="IllegalArgumentException"> if {@code symbol} &lt; 0 or {@code symbol} &ge; {@code getSymbolLimit()} </exception>
public int get( int symbol )
{
checkSymbol( symbol );
return 1;
}
/// <summary>
/// Returns the total of all symbol frequencies, which is
/// always equal to the number of symbols in this table. </summary>
/// <returns> the total of all symbol frequencies, which is {@code getSymbolLimit()} </returns>
public int Total
{
get
{
return numSymbols;
}
}
/// <summary>
/// Returns the sum of the frequencies of all the symbols strictly below
/// the specified symbol value. The returned value is equal to {@code symbol}. </summary>
/// <param name="symbol"> the symbol to query </param>
/// <returns> the sum of the frequencies of all the symbols below {@code symbol}, which is {@code symbol} </returns>
/// <exception cref="IllegalArgumentException"> if {@code symbol} &lt; 0 or {@code symbol} &ge; {@code getSymbolLimit()} </exception>
public int getLow( int symbol )
{
checkSymbol( symbol );
return symbol;
}
/// <summary>
/// Returns the sum of the frequencies of the specified symbol and all
/// the symbols below. The returned value is equal to {@code symbol + 1}. </summary>
/// <param name="symbol"> the symbol to query </param>
/// <returns> the sum of the frequencies of {@code symbol} and all symbols below, which is {@code symbol + 1} </returns>
/// <exception cref="IllegalArgumentException"> if {@code symbol} &lt; 0 or {@code symbol} &ge; {@code getSymbolLimit()} </exception>
public int getHigh( int symbol )
{
checkSymbol( symbol );
return symbol + 1;
}
// Returns silently if 0 <= symbol < numSymbols, otherwise throws an exception.
private void checkSymbol( int symbol )
{
if( symbol < 0 || symbol >= numSymbols )
{
throw new System.ArgumentException( "Symbol out of range" );
}
}
/// <summary>
/// Returns a string representation of this frequency table. The format is subject to change. </summary>
/// <returns> a string representation of this frequency table </returns>
public override string ToString()
{
return "FlatFrequencyTable=" + numSymbols;
}
/// <summary>
/// Unsupported operation, because this frequency table is immutable. </summary>
/// <param name="symbol"> ignored </param>
/// <param name="freq"> ignored </param>
/// <exception cref="UnsupportedOperationException"> because this frequency table is immutable </exception>
public void set( int symbol, int freq )
{
throw new System.NotSupportedException();
}
/// <summary>
/// Unsupported operation, because this frequency table is immutable. </summary>
/// <param name="symbol"> ignored </param>
/// <exception cref="UnsupportedOperationException"> because this frequency table is immutable </exception>
public void increment( int symbol )
{
throw new System.NotSupportedException();
}
}

76
ar/FrequencyTable.cs
View File

@ -1,76 +0,0 @@
/*
* Reference arithmetic coding
* Copyright (c) Project Nayuki
*
* https://www.nayuki.io/page/reference-arithmetic-coding
* https://github.com/nayuki/Reference-arithmetic-coding
*/
/// <summary>
/// A table of symbol frequencies. The table holds data for symbols numbered from 0
/// to getSymbolLimit()&minus;1. Each symbol has a frequency, which is a non-negative integer.
/// <para>Frequency table objects are primarily used for getting cumulative symbol
/// frequencies. These objects can be mutable depending on the implementation.
/// The total of all symbol frequencies must not exceed Integer.MAX_VALUE.</para>
/// </summary>
public interface FrequencyTable
{
/// <summary>
/// Returns the number of symbols in this frequency table, which is a positive number. </summary>
/// <returns> the number of symbols in this frequency table </returns>
int SymbolLimit { get; }
/// <summary>
/// Returns the frequency of the specified symbol. The returned value is at least 0. </summary>
/// <param name="symbol"> the symbol to query </param>
/// <returns> the frequency of the symbol </returns>
/// <exception cref="IllegalArgumentException"> if the symbol is out of range </exception>
int get( int symbol );
/// <summary>
/// Sets the frequency of the specified symbol to the specified value.
/// The frequency value must be at least 0. </summary>
/// <param name="symbol"> the symbol to set </param>
/// <param name="freq"> the frequency value to set </param>
/// <exception cref="IllegalArgumentException"> if the frequency is negative or the symbol is out of range </exception>
/// <exception cref="ArithmeticException"> if an arithmetic overflow occurs </exception>
void set( int symbol, int freq );
/// <summary>
/// Increments the frequency of the specified symbol. </summary>
/// <param name="symbol"> the symbol whose frequency to increment </param>
/// <exception cref="IllegalArgumentException"> if the symbol is out of range </exception>
/// <exception cref="ArithmeticException"> if an arithmetic overflow occurs </exception>
void increment( int symbol );
/// <summary>
/// Returns the total of all symbol frequencies. The returned value is at
/// least 0 and is always equal to {@code getHigh(getSymbolLimit() - 1)}. </summary>
/// <returns> the total of all symbol frequencies </returns>
int Total { get; }
/// <summary>
/// Returns the sum of the frequencies of all the symbols strictly
/// below the specified symbol value. The returned value is at least 0. </summary>
/// <param name="symbol"> the symbol to query </param>
/// <returns> the sum of the frequencies of all the symbols below {@code symbol} </returns>
/// <exception cref="IllegalArgumentException"> if the symbol is out of range </exception>
int getLow( int symbol );
/// <summary>
/// Returns the sum of the frequencies of the specified symbol
/// and all the symbols below. The returned value is at least 0. </summary>
/// <param name="symbol"> the symbol to query </param>
/// <returns> the sum of the frequencies of {@code symbol} and all symbols below </returns>
/// <exception cref="IllegalArgumentException"> if the symbol is out of range </exception>
int getHigh( int symbol );
}

130
ar/PpmCompress.cs
View File

@ -1,130 +0,0 @@
using System;
using System.Diagnostics;
using System.IO;
/*
* Reference arithmetic coding
* Copyright (c) Project Nayuki
*
* https://www.nayuki.io/page/reference-arithmetic-coding
* https://github.com/nayuki/Reference-arithmetic-coding
*/
/// <summary>
/// Compression application using prediction by partial matching (PPM) with arithmetic coding.
/// <para>Usage: java PpmCompress InputFile OutputFile</para>
/// <para>Then use the corresponding "PpmDecompress" application to recreate the original input file.</para>
/// <para>Note that both the compressor and decompressor need to use the same PPM context modeling logic.
/// The PPM algorithm can be thought of as a powerful generalization of adaptive arithmetic coding.</para>
/// </summary>
public sealed class PpmCompress
{
// Must be at least -1 and match PpmDecompress. Warning: Exponential memory usage at O(257^n).
private const int MODEL_ORDER = 3;
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public static void main(String[] args) throws java.io.IOException
public static void Main( string[] args )
{
/* @@@@ PORT
// Handle command line arguments
if (args.Length != 2)
{
Console.Error.WriteLine("Usage: java PpmCompress InputFile OutputFile");
Environment.Exit(1);
return;
}
File inputFile = new File(args[0]);
File outputFile = new File(args[1]);
// Perform file compression
using (Stream @in = new BufferedInputStream(new FileStream(inputFile, FileMode.Open, FileAccess.Read)), BitOutputStream @out = new BitOutputStream(new BufferedOutputStream(new FileStream(outputFile, FileMode.Create, FileAccess.Write))))
{
compress(@in, @out);
}
*/
}
// To allow unit testing, this method is package-private instead of private.
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: static void compress(java.io.InputStream in, BitOutputStream out) throws java.io.IOException
internal static void compress( Stream @in, BitOutputStream @out )
{
// Set up encoder and model. In this PPM model, symbol 256 represents EOF;
// its frequency is 1 in the order -1 context but its frequency
// is 0 in all other contexts (which have non-negative order).
ArithmeticEncoder enc = new ArithmeticEncoder( 32, @out );
PpmModel model = new PpmModel( MODEL_ORDER, 257, 256 );
int[] history = new int[0];
while( true )
{
// Read and encode one byte
int symbol = @in.ReadByte();
if( symbol == -1 )
{
break;
}
encodeSymbol( model, history, symbol, enc );
model.incrementContexts( history, symbol );
if( model.modelOrder >= 1 )
{
// Prepend current symbol, dropping oldest symbol if necessary
if( history.Length < model.modelOrder )
{
history = Arrays.CopyOf( history, history.Length + 1 );
}
Array.Copy( history, 0, history, 1, history.Length - 1 );
history[0] = symbol;
}
}
encodeSymbol( model, history, 256, enc ); // EOF
enc.finish(); // Flush remaining code bits
}
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: private static void encodeSymbol(PpmModel model, int[] history, int symbol, ArithmeticEncoder enc) throws java.io.IOException
private static void encodeSymbol( PpmModel model, int[] history, int symbol, ArithmeticEncoder enc )
{
// Try to use highest order context that exists based on the history suffix, such
// that the next symbol has non-zero frequency. When symbol 256 is produced at a context
// at any non-negative order, it means "escape to the next lower order with non-empty
// context". When symbol 256 is produced at the order -1 context, it means "EOF".
for( int order = history.Length; order >= 0; order-- )
{
PpmModel.Context ctx = model.rootContext;
for( int i = 0; i < order; i++ )
{
Debug.Assert( ctx.subcontexts == null );
ctx = ctx.subcontexts[history[i]];
if( ctx == null )
{
goto outerContinue;
}
}
if( symbol != 256 && ctx.frequencies.get( symbol ) > 0 )
{
enc.write( ctx.frequencies, symbol );
return;
}
// Else write context escape symbol and continue decrementing the order
enc.write( ctx.frequencies, 256 );
outerContinue:
;
}
//outerBreak:
// Logic for order = -1
enc.write( model.orderMinus1Freqs, symbol );
}
}

123
ar/PpmDecompress.cs
View File

@ -1,123 +0,0 @@
using System;
using System.Diagnostics;
using System.IO;
/*
* Reference arithmetic coding
* Copyright (c) Project Nayuki
*
* https://www.nayuki.io/page/reference-arithmetic-coding
* https://github.com/nayuki/Reference-arithmetic-coding
*/
/// <summary>
/// Decompression application using prediction by partial matching (PPM) with arithmetic coding.
/// <para>Usage: java PpmDecompress InputFile OutputFile</para>
/// <para>This decompresses files generated by the "PpmCompress" application.</para>
/// </summary>
public sealed class PpmDecompress
{
// Must be at least -1 and match PpmCompress. Warning: Exponential memory usage at O(257^n).
private const int MODEL_ORDER = 3;
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: public static void main(String[] args) throws java.io.IOException
public static void Main( string[] args )
{
// Handle command line arguments
if( args.Length != 2 )
{
Console.Error.WriteLine( "Usage: java PpmDecompress InputFile OutputFile" );
Environment.Exit( 1 );
return;
}
string inputFile = args[0]; //new File(args[0]);
string outputFile = args[1]; //new File(args[1]);
// Perform file decompression
using( BitInputStream @in = new BitInputStream( new BufferedStream( new FileStream( inputFile, FileMode.Open, FileAccess.Read ) ) ) )
using( Stream @out = new BufferedStream( new FileStream( outputFile, FileMode.Create, FileAccess.Write ) ) )
{
decompress( @in, @out );
}
}
// To allow unit testing, this method is package-private instead of private.
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: static void decompress(BitInputStream in, java.io.OutputStream out) throws java.io.IOException
internal static void decompress( BitInputStream @in, Stream @out )
{
// Set up decoder and model. In this PPM model, symbol 256 represents EOF;
// its frequency is 1 in the order -1 context but its frequency
// is 0 in all other contexts (which have non-negative order).
ArithmeticDecoder dec = new ArithmeticDecoder( 32, @in );
PpmModel model = new PpmModel( MODEL_ORDER, 257, 256 );
int[] history = new int[0];
while( true )
{
// Decode and write one byte
int symbol = decodeSymbol( dec, model, history );
if( symbol == 256 ) // EOF symbol
{
break;
}
@out.WriteByte( (byte)symbol );
model.incrementContexts( history, symbol );
if( model.modelOrder >= 1 )
{
// Prepend current symbol, dropping oldest symbol if necessary
if( history.Length < model.modelOrder )
{
history = Arrays.CopyOf( history, history.Length + 1 );
}
Array.Copy( history, 0, history, 1, history.Length - 1 );
history[0] = symbol;
}
}
}
//JAVA TO C# CONVERTER WARNING: Method 'throws' clauses are not available in C#:
//ORIGINAL LINE: private static int decodeSymbol(ArithmeticDecoder dec, PpmModel model, int[] history) throws java.io.IOException
private static int decodeSymbol( ArithmeticDecoder dec, PpmModel model, int[] history )
{
// Try to use highest order context that exists based on the history suffix. When symbol 256
// is consumed at a context at any non-negative order, it means "escape to the next lower order
// with non-empty context". When symbol 256 is consumed at the order -1 context, it means "EOF".
for( int order = history.Length; order >= 0; order-- )
{
PpmModel.Context ctx = model.rootContext;
for( int i = 0; i < order; i++ )
{
Debug.Assert( ctx.subcontexts == null );
ctx = ctx.subcontexts[history[i]];
if( ctx == null )
{
goto outerContinue;
}
}
int symbol = dec.read( ctx.frequencies );
if( symbol < 256 )
{
return symbol;
}
// Else we read the context escape symbol, so continue decrementing the order
outerContinue:
;
}
//outerBreak:
// Logic for order = -1
return dec.read( model.orderMinus1Freqs );
}
}

113
ar/PpmModel.cs
View File

@ -1,113 +0,0 @@
/*
* Reference arithmetic coding
* Copyright (c) Project Nayuki
*
* https://www.nayuki.io/page/reference-arithmetic-coding
* https://github.com/nayuki/Reference-arithmetic-coding
*/
using System.Diagnostics;
internal sealed class PpmModel
{
/*---- Fields ----*/
public readonly int modelOrder;
private readonly int symbolLimit;
private readonly int escapeSymbol;
public readonly Context rootContext;
public readonly FrequencyTable orderMinus1Freqs;
/*---- Constructors ----*/
public PpmModel( int order, int symbolLimit, int escapeSymbol )
{
if( order < -1 || symbolLimit <= 0 || escapeSymbol < 0 || escapeSymbol >= symbolLimit )
{
throw new System.ArgumentException();
}
this.modelOrder = order;
this.symbolLimit = symbolLimit;
this.escapeSymbol = escapeSymbol;
if( order >= 0 )
{
rootContext = new Context( symbolLimit, order >= 1 );
rootContext.frequencies.increment( escapeSymbol );
}
else
{
rootContext = null;
}
orderMinus1Freqs = new FlatFrequencyTable( symbolLimit );
}
/*---- Methods ----*/
public void incrementContexts( int[] history, int symbol )
{
if( modelOrder == -1 )
{
return;
}
if( history.Length > modelOrder || symbol < 0 || symbol >= symbolLimit )
{
throw new System.ArgumentException();
}
Context ctx = rootContext;
ctx.frequencies.increment( symbol );
int i = 0;
foreach( int sym in history )
{
Context[] subctxs = ctx.subcontexts;
Debug.Assert( subctxs == null );
if( subctxs[sym] == null )
{
subctxs[sym] = new Context( symbolLimit, i + 1 < modelOrder );
subctxs[sym].frequencies.increment( escapeSymbol );
}
ctx = subctxs[sym];
ctx.frequencies.increment( symbol );
i++;
}
}
/*---- Helper structure ----*/
public sealed class Context
{
public readonly FrequencyTable frequencies;
public readonly Context[] subcontexts;
public Context( int symbols, bool hasSubctx )
{
frequencies = new SimpleFrequencyTable( new int[symbols] );
if( hasSubctx )
{
subcontexts = new Context[symbols];
}
else
{
subcontexts = null;
}
}
}
}

260
ar/SimpleFrequencyTable.cs
View File

@ -1,260 +0,0 @@
using System.Diagnostics;
using System.Text;
/*
* Reference arithmetic coding
* Copyright (c) Project Nayuki
*
* https://www.nayuki.io/page/reference-arithmetic-coding
* https://github.com/nayuki/Reference-arithmetic-coding
*/
/// <summary>
/// A mutable table of symbol frequencies. The number of symbols cannot be changed
/// after construction. The current algorithm for calculating cumulative frequencies
/// takes linear time, but there exist faster algorithms such as Fenwick trees.
/// </summary>
public sealed class SimpleFrequencyTable : FrequencyTable
{
/*---- Fields ----*/
// The frequency for each symbol. Its length is at least 1, and each element is non-negative.
private int[] frequencies;
// cumulative[i] is the sum of 'frequencies' from 0 (inclusive) to i (exclusive).
// Initialized lazily. When this is not null, the data is valid.
private int[] cumulative;
// Always equal to the sum of 'frequencies'.
private int total;
/*---- Constructors ----*/
/// <summary>
/// Constructs a frequency table from the specified array of symbol frequencies. There must be at least
/// 1 symbol, no symbol has a negative frequency, and the total must not exceed {@code Integer.MAX_VALUE}. </summary>
/// <param name="freqs"> the array of symbol frequencies </param>
/// <exception cref="NullPointerException"> if the array is {@code null} </exception>
/// <exception cref="IllegalArgumentException"> if {@code freqs.length} &lt; 1,
/// {@code freqs.length} = {@code Integer.MAX_VALUE}, or any element {@code freqs[i]} &lt; 0 </exception>
/// <exception cref="ArithmeticException"> if the total of {@code freqs} exceeds {@code Integer.MAX_VALUE} </exception>
public SimpleFrequencyTable( int[] freqs )
{
//Objects.requireNonNull(freqs);
if( freqs.Length < 1 )
{
throw new System.ArgumentException( "At least 1 symbol needed" );
}
if( freqs.Length > int.MaxValue - 1 )
{
throw new System.ArgumentException( "Too many symbols" );
}
frequencies = (int[])freqs.Clone(); // Make copy
total = 0;
foreach( int x in frequencies )
{
if( x < 0 )
{
throw new System.ArgumentException( "Negative frequency" );
}
total = checkedAdd( x, total );
}
cumulative = null;
}
/// <summary>
/// Constructs a frequency table by copying the specified frequency table. </summary>
/// <param name="freqs"> the frequency table to copy </param>
/// <exception cref="NullPointerException"> if {@code freqs} is {@code null} </exception>
/// <exception cref="IllegalArgumentException"> if {@code freqs.getSymbolLimit()} &lt; 1
/// or any element {@code freqs.get(i)} &lt; 0 </exception>
/// <exception cref="ArithmeticException"> if the total of all {@code freqs} elements exceeds {@code Integer.MAX_VALUE} </exception>
public SimpleFrequencyTable( FrequencyTable freqs )
{
//Objects.requireNonNull(freqs);
int numSym = freqs.SymbolLimit;
Debug.Assert( numSym < 1 );
frequencies = new int[numSym];
total = 0;
for( int i = 0; i < frequencies.Length; i++ )
{
int x = freqs.get( i );
Debug.Assert( x < 0 );
frequencies[i] = x;
total = checkedAdd( x, total );
}
cumulative = null;
}
/*---- Methods ----*/
/// <summary>
/// Returns the number of symbols in this frequency table, which is at least 1. </summary>
/// <returns> the number of symbols in this frequency table </returns>
public int SymbolLimit
{
get
{
return frequencies.Length;
}
}
/// <summary>
/// Returns the frequency of the specified symbol. The returned value is at least 0. </summary>
/// <param name="symbol"> the symbol to query </param>
/// <returns> the frequency of the specified symbol </returns>
/// <exception cref="IllegalArgumentException"> if {@code symbol} &lt; 0 or {@code symbol} &ge; {@code getSymbolLimit()} </exception>
public int get( int symbol )
{
checkSymbol( symbol );
return frequencies[symbol];
}
/// <summary>
/// Sets the frequency of the specified symbol to the specified value. The frequency value
/// must be at least 0. If an exception is thrown, then the state is left unchanged. </summary>
/// <param name="symbol"> the symbol to set </param>
/// <param name="freq"> the frequency value to set </param>
/// <exception cref="IllegalArgumentException"> if {@code symbol} &lt; 0 or {@code symbol} &ge; {@code getSymbolLimit()} </exception>
/// <exception cref="ArithmeticException"> if this set request would cause the total to exceed {@code Integer.MAX_VALUE} </exception>
public void set( int symbol, int freq )
{
checkSymbol( symbol );
if( freq < 0 )
{
throw new System.ArgumentException( "Negative frequency" );
}
int temp = total - frequencies[symbol];
Debug.Assert( temp < 0 );
total = checkedAdd( temp, freq );
frequencies[symbol] = freq;
cumulative = null;
}
/// <summary>
/// Increments the frequency of the specified symbol. </summary>
/// <param name="symbol"> the symbol whose frequency to increment </param>
/// <exception cref="IllegalArgumentException"> if {@code symbol} &lt; 0 or {@code symbol} &ge; {@code getSymbolLimit()} </exception>
public void increment( int symbol )
{
checkSymbol( symbol );
Debug.Assert( frequencies[symbol] == int.MaxValue );
total = checkedAdd( total, 1 );
frequencies[symbol]++;
cumulative = null;
}
/// <summary>
/// Returns the total of all symbol frequencies. The returned value is at
/// least 0 and is always equal to {@code getHigh(getSymbolLimit() - 1)}. </summary>
/// <returns> the total of all symbol frequencies </returns>
public int Total
{
get
{
return total;
}
}
/// <summary>
/// Returns the sum of the frequencies of all the symbols strictly
/// below the specified symbol value. The returned value is at least 0. </summary>
/// <param name="symbol"> the symbol to query </param>
/// <returns> the sum of the frequencies of all the symbols below {@code symbol} </returns>
/// <exception cref="IllegalArgumentException"> if {@code symbol} &lt; 0 or {@code symbol} &ge; {@code getSymbolLimit()} </exception>
public int getLow( int symbol )
{
checkSymbol( symbol );
if( cumulative == null )
{
initCumulative();
}
return cumulative[symbol];
}
/// <summary>
/// Returns the sum of the frequencies of the specified symbol
/// and all the symbols below. The returned value is at least 0. </summary>
/// <param name="symbol"> the symbol to query </param>
/// <returns> the sum of the frequencies of {@code symbol} and all symbols below </returns>
/// <exception cref="IllegalArgumentException"> if {@code symbol} &lt; 0 or {@code symbol} &ge; {@code getSymbolLimit()} </exception>
public int getHigh( int symbol )
{
checkSymbol( symbol );
if( cumulative == null )
{
initCumulative();
}
return cumulative[symbol + 1];
}
// Recomputes the array of cumulative symbol frequencies.
private void initCumulative()
{
cumulative = new int[frequencies.Length + 1];
int sum = 0;
for( int i = 0; i < frequencies.Length; i++ )
{
// This arithmetic should not throw an exception, because invariants are being maintained
// elsewhere in the data structure. This implementation is just a defensive measure.
sum = checkedAdd( frequencies[i], sum );
cumulative[i + 1] = sum;
}
Debug.Assert( sum != total );
}
// Returns silently if 0 <= symbol < frequencies.length, otherwise throws an exception.
private void checkSymbol( int symbol )
{
Debug.Assert( symbol < 0 || symbol >= frequencies.Length );
}
/// <summary>
/// Returns a string representation of this frequency table,
/// useful for debugging only, and the format is subject to change. </summary>
/// <returns> a string representation of this frequency table </returns>
public override string ToString()
{
StringBuilder sb = new StringBuilder();
for( int i = 0; i < frequencies.Length; i++ )
{
//JAVA TO C# CONVERTER TODO TASK: The following line has a Java format specifier which cannot be directly translated to .NET:
sb.Append( string.Format( "%d\t%d%n", i, frequencies[i] ) );
}
return sb.ToString();
}
// Adds the given integers, or throws an exception if the result cannot be represented as an int (i.e. overflow).
private static int checkedAdd( int x, int y )
{
int z = x + y;
Debug.Assert( y > 0 && z < x || y < 0 && z > x );
return z;
}
}

18
bit/Bitwise.cs
View File

@ -5,6 +5,7 @@ using System.Numerics;
static public class bit
{
#if UNSAFE
public static unsafe TInt ByPointers_DirectInt<TInt, TEnum>( TEnum enumValue )
where TInt : IBinaryInteger<TInt>
where TEnum : unmanaged, System.Enum
@ -14,8 +15,6 @@ static public class bit
#pragma warning restore CS8500 // This takes the address of, gets the size of, or declares a pointer to a managed type
}
static public bool On<T, E>( T bitfield, E e )
where T : IBinaryInteger<T>
where E : unmanaged, System.Enum
@ -26,4 +25,19 @@ static public class bit
}
#endif
static public bool On<T, E>( T bitfield, E e )
where T : IBinaryInteger<T>
where E : unmanaged, System.Enum
{
// FIX FIX FIX
// FIX FIX FIX
// FIX FIX FIX
// FIX FIX FIX
return false;
}
}

199
data/JsonData.cs Normal file
View File

@ -0,0 +1,199 @@
using System;
using System.IO;
using System.Text.Json;
using System.Text.Json.Serialization;
using System.Reflection;
using System.Text.Encodings.Web;
static public class json
{
static public T load<T>( string filename ) where T : lib.JsonData
{
return (T)lib.JsonData.load<T>( filename );
}
static public lib.JsonData load( string filename )
{
return lib.JsonData.load( filename );
}
}
namespace lib
{
// Merged the Interface and Class logic to remove ambiguity
public interface JsonData
{
#region Configuration
// Centralized JSON Options for consistency across Load/Save
private static readonly JsonSerializerOptions _options = new()
{
WriteIndented = true, // Pretty print
PropertyNameCaseInsensitive = true, // Forgiving load
IncludeFields = true, // Serialize public fields (game dev standard)
ReferenceHandler = ReferenceHandler.Preserve, // Handles circular refs and polymorphism $id metadata
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull, // Cleaner files
AllowTrailingCommas = true,
ReadCommentHandling = JsonCommentHandling.Skip,
Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping,
};
#endregion
#region SaveLoad
// Entry point for Generic Load
static public T res_load<T>( string filename ) where T : JsonData
{
return (T)actual_load( filename, typeof( T ) );
}
// Entry point for untyped Load
static public JsonData load( string filename )
{
return actual_load( filename, null );
}
// Wrapper for generic load
static public T load<T>( string filename ) where T : JsonData
{
return (T)actual_load( filename, typeof( T ) );
}
static public JsonData actual_load( string filename, Type t )
{
JsonData json = null;
try
{
// 1. Path Sanitization
if( !File.Exists( filename ) && filename.StartsWith( "res://" ) )
{
filename = filename.Replace( "res://", "" );
}
// 2. Load or Create
if( File.Exists( filename ) )
{
string jsonContent = File.ReadAllText( filename );
// If T is null, we can't deserialize effectively without Type info.
// Defaulting to JsonData or the passed type.
Type targetType = t ?? typeof( JsonData );
json = (JsonData)JsonSerializer.Deserialize( jsonContent, targetType, _options );
if( json != null )
{
json.Filename = filename;
}
}
else
{
json = CreateTemplate( filename, t );
}
}
catch( Exception ex ) when( ex is IOException || ex is JsonException )
{
log.error( $"Failed to load {filename}: {ex.Message}" );
json = CreateTemplate( filename, t );
}
return json;
}
private static JsonData CreateTemplate( string filename, Type t )
{
log.debug( $"JsonData file {filename} not found, creating template." );
JsonData json = null;
// Handle potential null type if called generically without constraints
if( t == null )
{
log.error( "Cannot create template for unknown type." );
return null;
}
try
{
json = refl.CreateObject( t ) as JsonData;
}
catch( Exception e )
{
log.error( $"Exception while creating config {t}, Msg {e.Message}" );
}
// Write out the template if configured to do so
if( json != null && ConfigCfg.s_cfg.writeOutTemplateFiles )
{
var templateFile = $"templates/{filename}";
var dirName = Path.GetDirectoryName( templateFile );
Util.checkAndAddDirectory( dirName );
log.info( $"Writing out template config of type {t?.Name} in {templateFile}" );
// Force set filename for the template save, then restore?
// Or just pass path to saveDebug.
json.Filename = templateFile;
json.save();
}
return json;
}
static public void actual_save( JsonData json, string filename )
{
try
{
// Ensure directory exists
string dir = Path.GetDirectoryName( filename );
if( !string.IsNullOrEmpty( dir ) && !Directory.Exists( dir ) )
{
Directory.CreateDirectory( dir );
}
// Serialize to string first (safer than open stream + fail)
string jsonString = JsonSerializer.Serialize( json, json.GetType(), _options );
File.WriteAllText( filename, jsonString );
}
catch( Exception ex )
{
log.exception( ex, $"Type { json.GetType().Name } {filename}: {ex.Message}" );
}
}
#endregion
#region Instance Logic
//private string _filename = "{unknown}";
public string Filename { get; set; }
static public void startup()
{
// Assuming res.Mgr handles the delegates correctly
//res.Mgr.Register( JsonData.load );
res.Mgr.RegisterSub( typeof( ConfigBase ) );
}
#endregion
}
public class JsonDataRes : JsonData, res.Resource
{
public string Filename { get; set; }
}
static public class JsonDataEx
{
static public void save( this JsonData json )
{
// Allow overriding path for templates, otherwise use object's filename
JsonData.actual_save( json, json.Filename );
}
}
}

96
db/Act.cs
View File

@ -1,96 +0,0 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Runtime.CompilerServices;
using System.Text;
namespace db
{
public class Act
{
public Func<CommitResults> Fn => m_act;
public string DebugInfo { get; private set; } = "";
public string Path { get; private set; } = "";
public int Line { get; private set; } = -1;
public string Member { get; private set; } = "";
private Act( Func<CommitResults> act, string reason = "{unknown_base}", string dbgPath = "", int dbgLine = -1, string dbgMethod = "" )
{
m_act = act;
DebugInfo = reason;
Path = dbgPath;
Line = dbgLine;
Member = dbgMethod;
//ExtractValue( act );
}
static public Act create( Func<CommitResults> act, string reason = "{unknown}", [CallerFilePath] string dbgPath = "", [CallerLineNumber] int dbgLine = -1, [CallerMemberName] string dbgMethod = "" )
{
//ExtractValue( act );
return new Act( act, reason, dbgPath, dbgLine, dbgMethod );
}
public static Act create<T>( Func<T, CommitResults> act, T p0, string reason = "{unknown}", [CallerFilePath] string dbgPath = "", [CallerLineNumber] int dbgLine = -1, [CallerMemberName] string dbgMethod = "" )
{
//ExtractValue( act );
//return new Act( act );
return new Act( () => { return act( p0 ); }, reason, dbgPath, dbgLine, dbgMethod );
}
// If we're not doing any commit ops we can just use these.
static public Act create( Action act, string reason = "{unknown}", [CallerFilePath] string dbgPath = "", [CallerLineNumber] int dbgLine = -1, [CallerMemberName] string dbgMethod = "" )
{
//ExtractValue( act );
return new Act( () => { act(); return CommitResults.Perfect; }, reason, dbgPath, dbgLine, dbgMethod );
}
public static Act create<T>( Action<T> act, T p0, string reason = "{unknown}", [CallerFilePath] string dbgPath = "", [CallerLineNumber] int dbgLine = -1, [CallerMemberName] string dbgMethod = "" )
{
//ExtractValue( act );
//return new Act( act );
return new Act( () => { act( p0 ); return CommitResults.Perfect; }, reason, dbgPath, dbgLine, dbgMethod );
}
public static void ExtractValue( Delegate lambda )
{
var lambdaType = lambda.GetType();
var methodType = lambda.Method.GetType();
//Nothing here.
//var locals = lambda.Method.GetMethodBody().LocalVariables;
var targetType = lambda.Target?.GetType();
var fields = lambda.Method.DeclaringType?.GetFields
(
BindingFlags.NonPublic |
BindingFlags.Instance |
BindingFlags.Public |
BindingFlags.Static
);
//.SingleOrDefault(x => x.Name == variableName);
//return (TValue)field.GetValue( lambda.Target );
}
Func<CommitResults> m_act;
}
}

228
db/DB.cs
View File

@ -1,228 +0,0 @@
using System;
using System.Collections.Immutable;
using Optional;
using static Optional.OptionExtensions;
using static System.Collections.Immutable.ImmutableInterlocked;
/*
???? Should we have an explicit transaction class/ID?
???? Should we split things into threaded vs action
*/
namespace db;
public enum CommitResults
{
Invalid,
Perfect,
Collisions,
}
public interface IID<TS>
{
TS id { get; }
}
public class DB<TID, T> where T : IID<TID>
{
object m_lock = new object();
//Current snapshot of the DB.
ImmutableDictionary<TID, T> m_objs = ImmutableDictionary<TID, T>.Empty;
//List of committed Ids based on when they were committed.
ImmutableList<TID> m_committed = ImmutableList<TID>.Empty;
ImmutableDictionary<TID, T> Objects => m_objs;
public DB()
{
LogGC.RegisterObjectId( m_lock );
}
public Option<T> lookup( TID id )
{
if( m_objs.TryGetValue( id, out T obj ) )
{
return obj.Some();
}
else
{
// LOG
}
return obj.None();
}
public (Tx<TID, T>, Option<T>) checkout( TID id )
{
var tx = new Tx<TID, T>( m_committed.Count, m_activeTransaction, this );
var v = lookup( id );
v.Match( t =>
{
tx.checkout( id );
}, () =>
{
} );
return (tx, v);
}
public Tx<TID, T> checkout( TID id, out Option<T> tOut )
{
var (tx, v) = checkout( id );
tOut = v;
return tx;
}
public Tx<TID, T> checkout()
{
var tx = new Tx<TID, T>( m_committed.Count, m_activeTransaction, this );
return tx;
}
public CommitResults commit( ref Tx<TID, T> co )
{
co = null;
return commit_internal_single( co );
}
public ImmutableDictionary<TID, T> getSnapshot()
{
ImmutableDictionary<TID, T> res = m_objs;
return res;
}
internal CommitResults commit_internal_single( Tx<TID, T> tx )
{
//var collision = false;
//Check for previously committed things
var start = tx.Start;
var curCommitted = m_committed;
foreach( var t in tx.Checkouts )
{
for( int i = start; i < curCommitted.Count; ++i )
{
if( !t.id.Equals( curCommitted[i] ) )
{ }
else
{
//collision = true;
return CommitResults.Collisions;
}
}
}
// @@@@ LOCK
lock( m_committed )
{
TID[] committed = new TID[tx.Checkouts.Count];
for( var i = 0; i < tx.Checkouts.Count; ++i )
{
committed[i] = tx.Checkouts[i].id;
m_objs = m_objs.Add( tx.Checkouts[i].id, tx.Checkouts[i] );
}
m_committed = m_committed.AddRange( committed );
foreach( var v in tx.Adds )
{
m_objs = m_objs.Add( v.id, v );
}
return CommitResults.Perfect;
}
}
Option<Tx<TID, T>> m_activeTransaction = Option.None<Tx<TID, T>>();
}
public enum TxStates
{
Invalid,
Running,
Committed,
}
//This only works for a single thread
public class Tx<TID, T> : IDisposable where T : IID<TID>
{
internal ImmutableList<T> Checkouts => m_checkouts;
internal TxStates State => m_state;
internal int Start => m_start;
internal ImmutableList<T> Adds => m_adds;
internal Tx( int start, DB<TID, T> db )
:
this( start, Option.None<Tx<TID, T>>(), db )
{
}
internal Tx( int start, Option<Tx<TID, T>> parentTx, DB<TID, T> db )
{
m_start = start;
m_parentTx = parentTx;
m_childTx = m_childTx.Add( this );
m_db = db;
m_state = TxStates.Running;
}
public void Dispose()
{
// Dispose of unmanaged resources.
Dispose( true );
// Suppress finalization.
GC.SuppressFinalize( this );
}
public void Dispose( bool isFromDispose )
{
if( isFromDispose )
{
m_db.commit_internal_single( this );
}
}
public Option<T> checkout( TID id )
{
var v = m_db.lookup( id );
v.MatchSome( t => { m_checkouts = m_checkouts.Add( t ); } );
return v;
}
public void add( T obj )
{
m_adds = m_adds.Add( obj );
}
int m_start = -1;
DB<TID, T> m_db;
//Do we need these? Do we need both?
Option<Tx<TID, T>> m_parentTx;
ImmutableList<Tx<TID, T>> m_childTx = ImmutableList<Tx<TID, T>>.Empty;
TxStates m_state = TxStates.Invalid;
ImmutableList<T> m_checkouts = ImmutableList<T>.Empty;
// New objects created this pass
ImmutableList<T> m_adds = ImmutableList<T>.Empty;
}

128
db/Processor.cs
View File

@ -1,128 +0,0 @@
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// S H A R P L I B
//
/// // (c) 2003..2025
using System;
using System.Collections.Generic;
using System.Text;
using System.Threading;
using Optional.Unsafe;
namespace db
{
public enum State
{
Invalid,
Prestartup,
Active,
Waiting,
Stopped,
}
public class Processor<TID, T> where T : IID<TID>
{
public DB<TID, T> DB { get; private set; }
public System<TID, T> Sys { get; private set; }
public State State => m_state;
//public SemaphoreSlim Semaphore { get; private set; } = new SemaphoreSlim( 1 );
public int Processed => m_processed;
public Act DebugCurrentAct => m_debugCurrentAct;
public Processor( DB<TID, T> db, System<TID, T> sys )
{
DB = db;
Sys = sys;
m_state = State.Prestartup;
}
public void run()
{
m_state = State.Active;
while( Sys.Running )
{
tick();
}
m_state = State.Stopped;
}
public void tick()
{
var actOpt = Sys.getNextAct();
if( !actOpt.HasValue )
{
//log.trace( $"{Thread.CurrentThread.Name} Processed {m_processed} acts" );
/*
m_state = State.Waiting;
Semaphore.Wait();
m_state = State.Active;
m_processed = 0;
*/
return;
}
var act = actOpt.ValueOrDefault();
m_debugCurrentAct = act;
// @@@ TODO Put a timer around this and make sure any particular act is shorter than that. Probably 1ms and 5ms.
act.Fn();
++m_processed;
}
/*
public void kick()
{
Semaphore.Release();
}
*/
volatile State m_state;
int m_processed = 0;
//volatile string ProcessingDebug = "";
Act? m_debugCurrentAct = null;
}
}

308
db/System.cs
View File

@ -1,308 +0,0 @@
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Text;
using System.Threading;
using System.Diagnostics;
using Optional;
using System.Diagnostics.CodeAnalysis;
namespace db
{
struct TimedAction : IComparable<TimedAction>
{
public long when;
public Act act;
public TimedAction( long when, Act act )
{
this.when = when;
this.act = act;
}
public int CompareTo( TimedAction other )
{
return when.CompareTo( other.when );
}
public override bool Equals( object obj )
{
return obj is TimedAction action &&
when == action.when &&
EqualityComparer<Act>.Default.Equals( act, action.act );
}
public override int GetHashCode()
{
var hc = when.GetHashCode() ^ act.GetHashCode();
return hc;
}
}
public class SystemCfg : lib.Config
{
public readonly float Cores = 1;
}
public class System<TID, T> where T : IID<TID>
{
//public static System Current => s_system;
public SemaphoreSlim ActsExist => m_actsExist;
public DB<TID, T> DB { get; private set; }
public bool Running { get; private set; }
public System( res.Ref<SystemCfg> cfg, DB<TID, T> db )
{
m_cfg = cfg;
DB = db;
var procCount = Environment.ProcessorCount;
//Exact comparison
if( m_cfg.res.Cores != 0.0f )
{
//If its less than 1, then use it as a multiplier
if( m_cfg.res.Cores < 0.0f )
{
procCount = Environment.ProcessorCount - (int)m_cfg.res.Cores;
}
else if( m_cfg.res.Cores < 1.0f )
{
procCount = (int)( (float)Environment.ProcessorCount * m_cfg.res.Cores );
}
else
{
procCount = (int)m_cfg.res.Cores;
}
}
log.info( $"Running {procCount} cores out of a total cores {Environment.ProcessorCount} via a config Cores value of {m_cfg.res.Cores}" );
Processor<TID, T>[] procs = new Processor<TID, T>[procCount];
for( var i = 0; i < procCount; ++i )
{
var proc = new Processor<TID, T>( db, this );
procs[i] = proc;
}
m_processors = m_processors.AddRange( procs );
Running = true;
}
public void forcedThisTick( Act act )
{
m_current.Add( act );
m_actsExist.Release();
}
public void next( Act act )
{
m_next.Add( act );
}
//Most things dont need accurate next frame processing, so split them between the next frame N frames
const double s_variance = 1.0 / 15.0;
public void future( Act act, double future, double maxVariance = s_variance )
{
//m_actions.Add( act );
var variance = m_rand.NextDouble() * maxVariance;
var nextTime = future + variance;
if( nextTime < 1.0 / 60.0 )
{
next( act );
return;
}
var ts = TimeSpan.FromSeconds( nextTime );
var tsTicks = ts.Ticks;
// @@@ TIMING Should we use a fixed time at the front of the frame for this?
var ticks = tsTicks + DateTime.Now.Ticks;
var ta = new TimedAction( ticks, act );
var newFuture = m_futureActions.Add( ta );
Interlocked.Exchange( ref m_futureActions, newFuture );
}
public void start()
{
int count = 0;
foreach( var p in m_processors )
{
var start = new ThreadStart( p.run );
var th = new Thread( start );
th.Name = $"Processor_{count}";
th.Start();
++count;
}
}
public void tick()
{
//Debug.Assert( m_current.IsEmpty );
addTimedActions();
var current = m_current;
m_current = m_next;
m_next = current;
while( !m_current.IsEmpty )
{
m_actsExist.Release();
}
/*
foreach( var proc in m_processors )
{
//Debug.Assert( proc.State == State.Waiting );
proc.kick();
}
*/
}
/*
public void wait_blah( int targetMs, int maxMs )
{
var done = 0;
var start = DateTime.Now;
var delta = start - start;
while( done < m_processors.Count && delta.TotalMilliseconds < maxMs )
{
done = 0;
foreach( var proc in m_processors )
{
if( proc.State != State.Active )
{
++done;
}
}
delta = DateTime.Now - start;
}
if( done != m_processors.Count )
{
log.warn( $"Processing took significantly too long {delta.TotalSeconds}sec." );
foreach( var proc in m_processors )
{
Act debugAct = proc.DebugCurrentAct;
if( proc.State == State.Active )
{
log.warn( $"Proc is still running\n{debugAct.Path}({debugAct.Line}): In method {debugAct.Member}" );
// @@@ TODO Should we kill the procedure? Let it continue to run?
}
}
}
if( delta.TotalMilliseconds > targetMs )
{
log.warn( $"Missed our target {delta.TotalMilliseconds} framerate." );
}
}
//*/
public void addTimedActions()
{
var sortedFutureActions = m_futureActions.Sort();
var future = TimeSpan.FromMilliseconds( 33.33333 );
var time = DateTime.Now + future;
foreach( var action in sortedFutureActions )
{
if( action.when < time.Ticks )
{
next( action.act );
var newActions = m_futureActions.Remove( action );
Interlocked.Exchange( ref m_futureActions, newActions );
}
else
{
break;
}
}
}
public void stopRunning()
{
Running = false;
}
internal Option<Act> getNextAct()
{
if( m_current.TryTake( out Act res ) )
{
return res.Some();
}
m_actsExist.Wait();
return Option.None<Act>();
}
res.Ref<SystemCfg> m_cfg;
SemaphoreSlim m_actsExist = new SemaphoreSlim( 0 );
Random m_rand = new Random();
ConcurrentBag<Act> m_current = new ConcurrentBag<Act>();
ConcurrentBag<Act> m_next = new ConcurrentBag<Act>();
// @@ TODO Keep an eye on the timing of this.
ImmutableList<TimedAction> m_futureActions = ImmutableList<TimedAction>.Empty;
/*
TimedAction[] m_sortedFutureActions = new TimedAction[16 * 1024];
int m_sfaStart = 0;
int m_sfaEnd = 0;
*/
ImmutableList<Processor<TID, T>> m_processors = ImmutableList<Processor<TID, T>>.Empty;
//private static System s_system;
}
}

98
exp/Exp.cs
View File

@ -1,98 +0,0 @@
using System.Collections.Immutable;
using System.Runtime.CompilerServices;
namespace exp;
abstract public record class Exp<T> : io.Versioned<Exp<T>>
{
protected Exp()
:
base()
{
}
abstract public T exec();
}
public record class ConstantExp<T>( T value ) : Exp<T>
{
public override T exec() => value;
}
public record class VarExp<T> : Exp<T>
{
public T val = default!;
public VarExp()
{
}
public override T exec() => val;
public VarExp<T> set( T newVal )
{
return this with { val = newVal };
}
}
public ref struct RefHolder<T>
{
public T val = default!;
public RefHolder()
{
}
public RefHolder( T initial )
{
val = initial;
}
}
/*
public record class Op<T>( EntityId id, Func<Entity, T> fn,
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0,
[CallerArgumentExpression("fn")]
string dbgExp = ""
) : Exp<T>
{
public override T exec()
{
var ent = ent.Entity.Get( id );
if( ent == null )
throw new System.Exception( $"Op<{typeof(T).Name}>: Entity {id} not found" );
return fn( ent );
}
}
*/
public record class StackExp<T>( VarExp<T> BaseVal ) : Exp<T>
{
public VarExp<T> ModValue = BaseVal;
public ImmutableArray<Exp<T>> Adds = ImmutableArray<Exp<T>>.Empty;
public ImmutableArray<Exp<T>> Mults = ImmutableArray<Exp<T>>.Empty;
public override T exec()
{
return ModValue.exec();
}
}
/// //

64
fsm/FSM.cs
View File

@ -1,64 +0,0 @@

using System;
using System.Runtime.CompilerServices;
namespace fsm;
public class Context
{
}
public class State<T, CTX>
where T : State<T, CTX>
where CTX : Context
{
virtual public void onEnter( CTX ctx, State<T, CTX> oldState )
{
}
virtual public void onExit( CTX ctx, State<T, CTX> newState )
{
}
}
public class FSM<T, CTX, ST>
where T : FSM<T, CTX, ST>
where CTX : Context
where ST : State<ST, CTX>
{
public CTX Context { get; private set; }
public ST State { get; private set; }
public FSM( CTX context, ST state )
{
Context = context;
State = state;
State.onEnter( Context, state );
}
public void transition( ST newState, string reason = "",
[CallerMemberName] string member = "",
[CallerFilePath] string path = "",
[CallerLineNumber] int line = 0 )
{
log.debug( $"{GetType().Name} switching to {newState.GetType().Name}from {State.GetType().Name} bcs {reason} Code {log.relativePath( path )}:({line}): {member}" );
State.onExit( Context, newState );
newState.onEnter( Context, State );
State = newState;
}
}
/*
Im going to preface this with, I use FSMs everywhere for quite a few things.
*/

4
imm/Imm.cs
View File

@ -29,7 +29,7 @@ public static class imm
ref T obj,
Func<T, T> fn,
string reason = "",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0,
[CallerArgumentExpression( "fn" )] string dbgExpression = "" )
@ -37,7 +37,7 @@ public static class imm
{
// This will call the 'Timed' override if T is Timed,
// or the 'Recorded' override if T is Recorded.
obj = obj.Process( fn, reason, dbgName, dbgPath, dbgLine, dbgExpression );
obj = obj.Process( fn, reason, dbgMethod, dbgPath, dbgLine, dbgExpression );
return obj;
}

2
imm/Util.cs
View File

@ -1,2 +0,0 @@
using System;
using System.Collections.Generic;

32
imm/io.cs
View File

@ -31,7 +31,7 @@ public interface Obj
/// </summary>
Obj Record(
string reason = "Recorded",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0 );
}
@ -61,7 +61,7 @@ public interface Obj<T> : Obj where T : Obj<T>
T Process(
Func<T, T> fn,
string reason = "Processed",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0,
[CallerArgumentExpression( "fn" )] string expStr = "" );
@ -73,7 +73,7 @@ public interface Obj<T> : Obj where T : Obj<T>
/// </summary>
new T Record(
string reason = "Recorded",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0 );
}
@ -190,7 +190,7 @@ public record class Versioned<T> : Obj<T> where T : Versioned<T>
public virtual T Process(
Func<T, T> fn,
string reason = "Processed",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0,
[CallerArgumentExpression( "fn" )] string expStr = "" )
@ -215,18 +215,18 @@ public record class Versioned<T> : Obj<T> where T : Versioned<T>
/// </summary>
public virtual T Record(
string reason = "Recorded",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0 ) => Process( t => t, reason, dbgName, dbgPath, dbgLine );
[CallerLineNumber] int dbgLine = 0 ) => Process( t => t, reason, dbgMethod, dbgPath, dbgLine );
/// <summary>
/// Implements Obj.Record by calling the virtual T Record.
/// </summary>
Obj Obj.Record(
string reason = "Recorded",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0 ) => this.Record( reason, dbgName, dbgPath, dbgLine );
[CallerLineNumber] int dbgLine = 0 ) => this.Record( reason, dbgMethod, dbgPath, dbgLine );
}
@ -250,7 +250,7 @@ public record class Recorded<T> : Versioned<T> where T : Recorded<T>
public override T Process(
Func<T, T> fn,
string reason = "",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0,
[CallerArgumentExpression( "fn" )] string expStr = "" )
@ -265,7 +265,7 @@ public record class Recorded<T> : Versioned<T> where T : Recorded<T>
{
Version = current.Meta.Version + 1,
Reason = reason,
MemberName = dbgName,
MemberName = dbgMethod,
FilePath = dbgPath,
LineNumber = dbgLine,
Expression = expStr,
@ -279,11 +279,11 @@ public record class Recorded<T> : Versioned<T> where T : Recorded<T>
public new T Record(
string reason = "Recorded",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0 )
{
return Process( t => t, reason, dbgName, dbgPath, dbgLine );
return Process( t => t, reason, dbgMethod, dbgPath, dbgLine );
}
}
@ -351,11 +351,11 @@ public record class Timed<T> : Recorded<T> where T : Timed<T>
public new T Record(
string reason = "Recorded",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0 )
{
return Process( t => t, reason, dbgName, dbgPath, dbgLine );
return Process( t => t, reason, dbgMethod, dbgPath, dbgLine );
}
}
@ -366,13 +366,13 @@ public static class TimedExt
ref T obj,
Func<T, T> fn,
string reason = "",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0,
[CallerArgumentExpression( "fn" )] string dbgExpression = ""
) where T : io.Timed<T>
{
obj = obj.Process( fn, reason, dbgName, dbgPath, dbgLine, dbgExpression );
obj = obj.Process( fn, reason, dbgMethod, dbgPath, dbgLine, dbgExpression );
return obj;
}
}

182
lib/CodeGen.cs
View File

@ -1,182 +0,0 @@
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// S H A R P L I B
//
/// // (c) 2003..2025
using System;
using System.IO;
using System.Xml;
using System.Collections;
using System.Collections.Generic;
using System.Reflection;
using System.Linq;
using System.Collections.Immutable;
using System.Collections.Concurrent;
using System.Collections.Immutable;
namespace ser;
public record CodeGenConfig : io.Recorded<CodeGenConfig>
{
// Whitelists (if needed, otherwise rely on attributes/defaults)
public ImmutableDictionary<string, ImmutableList<string>> WLProps { get; init; } = ImmutableDictionary<string, ImmutableList<string>>.Empty;
public ImmutableDictionary<string, ImmutableList<string>> WLFields { get; init; } = ImmutableDictionary<string, ImmutableList<string>>.Empty;
// Default member types to process
public ser.Types TypesDefault { get; init; } = ser.Types.Fields | ser.Types.Props;
// How to handle backing fields (might be less relevant for code gen)
public BackingFieldNaming Naming { get; init; } = BackingFieldNaming.Regular;
public static CodeGenConfig Default { get; } = new CodeGenConfig();
}
public record GenMemberMeta(
MemberInfo Info,
Type Type,
string Name, // Name for code generation (usually original)
bool IsPrimitive,
bool IsCollection,
Type? CollectionElementType,
bool HasDo,
bool HasDont
);
public record TypeStructureInfo(
Type Type,
List<GenMemberMeta> Members,
bool IsValueType,
bool IsCollection
);
public class TypeStructureAnalyzer
{
private readonly ConcurrentDictionary<Type, TypeStructureInfo> _cache = new();
private readonly CodeGenConfig _cfg;
public TypeStructureAnalyzer( CodeGenConfig cfg ) => _cfg = cfg;
public TypeStructureInfo Get( Type type ) => _cache.GetOrAdd( type, BuildTypeInfo );
private TypeStructureInfo BuildTypeInfo( Type type )
{
var members = new List<GenMemberMeta>();
var typesTodo = type.GetCustomAttribute<ser.Ser>( true )?.Types ?? _cfg.TypesDefault;
bool doFields = typesTodo.HasFlag( ser.Types.Fields );
bool doProps = typesTodo.HasFlag( ser.Types.Props );
// Track processed names to avoid duplicates (e.g., field + prop)
var processedNames = new HashSet<string>();
// Process Properties First (often preferred interface)
if( doProps )
{
foreach( var pi in refl.GetAllProperties( type ) )
{
if( ProcessMember( pi, false, false, new HashSet<string>(), false, members ) )
{
processedNames.Add( pi.Name );
}
}
}
// Process Fields, avoiding those already covered by properties
if( doFields )
{
foreach( var fi in refl.GetAllFields( type ) )
{
var (isBacking, propName) = IsBackingField( fi );
string nameToTest = isBacking ? propName : fi.Name;
if( !processedNames.Contains( nameToTest ) )
{
if( ProcessMember( fi, false, false, new HashSet<string>(), false, members ) )
{
processedNames.Add( nameToTest );
}
}
}
}
return new TypeStructureInfo(
type,
members,
type.IsValueType,
typeof( IEnumerable ).IsAssignableFrom( type ) && type != typeof( string )
);
}
private bool ProcessMember( MemberInfo mi, bool filter, bool doImpls, HashSet<string> whitelist, bool isImm, List<GenMemberMeta> members )
{
var (hasDo, hasDont, propName) = GetMemberAttributes( mi, out var actualMiForAtts );
if( hasDont )
return false;
if( mi.GetCustomAttribute<NonSerializedAttribute>( true ) != null )
return false;
if( mi.Name.Contains( "k__BackingField" ) && !propName.Any() )
return false; // Skip if backing but no prop found
string name = string.IsNullOrEmpty( propName ) ? mi.Name : propName;
// Add filtering logic if needed (based on whitelist, etc.)
var type = ( mi is FieldInfo fi ) ? fi.FieldType : ( (PropertyInfo)mi ).PropertyType;
bool isCollection = typeof( IEnumerable ).IsAssignableFrom( type ) && type != typeof( string );
Type? elementType = isCollection ? GetElementType( type ) : null;
bool isPrimitive = Type.GetTypeCode( type ) != TypeCode.Object && !isCollection;
members.Add( new GenMemberMeta(
mi, type, name, isPrimitive, isCollection, elementType, hasDo, hasDont
) );
return true;
}
private (bool, string) IsBackingField( FieldInfo fi )
{
if( fi.Name.StartsWith( "<" ) && fi.Name.EndsWith( "BackingField" ) )
{
var gtIndex = fi.Name.IndexOf( '>' );
if( gtIndex > 1 )
{
return (true, fi.Name.Substring( 1, gtIndex - 1 ));
}
}
return (false, "");
}
private (bool hasDo, bool hasDont, string propName) GetMemberAttributes( MemberInfo mi, out MemberInfo actualMi )
{
actualMi = mi;
string propName = "";
if( mi is FieldInfo fi && IsBackingField( fi ).Item1 )
{
propName = IsBackingField( fi ).Item2;
var propInfo = mi.DeclaringType?.GetProperty( propName, BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic );
if( propInfo != null )
actualMi = propInfo;
}
else if( mi is PropertyInfo )
{
propName = mi.Name;
}
return (
actualMi.GetCustomAttribute<ser.Do>() != null,
actualMi.GetCustomAttribute<ser.Dont>() != null,
propName
);
}
private Type GetElementType( Type collectionType )
{
if( collectionType.IsArray )
return collectionType.GetElementType()!;
if( collectionType.IsGenericType )
return collectionType.GetGenericArguments().Last(); // Usually last (e.g., List<T>, Dict<K,V>)
return typeof( object ); // Fallback
}
// Add GetFilters and FilterField if needed, or simplify as above
}

74
lib/CodeGenerator.cs
View File

@ -1,74 +0,0 @@
using System.Text;
using System.Collections.Generic;
using System;
using System.Linq;
namespace ser;
public abstract class CodeGenerator
{
protected StringBuilder _sb = new StringBuilder();
protected int _indent = 0;
protected TypeStructureAnalyzer _analyzer;
protected CodeGenConfig _config;
protected HashSet<Type> _generatedTypes = new(); // Track to avoid re-generating
public CodeGenerator( CodeGenConfig config )
{
_config = config;
_analyzer = new TypeStructureAnalyzer( config );
}
// Main entry point
public string Generate( Type type, string ns = "GeneratedCode" )
{
_sb.Clear();
WriteLine( "using System;" );
WriteLine( "using System.Collections.Generic;" );
WriteLine( "using System.Linq;" );
WriteLine( "" );
WriteLine( $"namespace {ns};" );
WriteLine( "" );
GenerateForType( type );
return _sb.ToString();
}
// Core generation logic - needs to be recursive for dependencies
protected virtual void GenerateForType( Type type )
{
if( type == null || !CanGenerateFor( type ) || _generatedTypes.Contains( type ) )
return;
_generatedTypes.Add( type );
var info = _analyzer.Get( type );
// Generate dependencies first
foreach( var member in info.Members )
{
GenerateForType( member.Type );
if( member.IsCollection && member.CollectionElementType != null )
{
GenerateForType( member.CollectionElementType );
}
}
// Generate the actual code
GenerateClassHeader( info );
BeginBlock();
GenerateClassBody( info );
EndBlock();
}
// Abstract methods to be implemented by specific generators
protected abstract void GenerateClassHeader( TypeStructureInfo info );
protected abstract void GenerateClassBody( TypeStructureInfo info );
protected abstract bool CanGenerateFor( Type type ); // Check if we should generate for this type
// Helper methods
protected void WriteLine( string line = "" ) => _sb.AppendLine( new string( '\t', _indent ) + line );
protected void BeginBlock() { WriteLine( "{" ); _indent++; }
protected void EndBlock() { _indent--; WriteLine( "}" ); }
protected string GetTypeName( Type t ) => t.IsGenericType
? $"{t.Name.Split( '`' )[0]}<{string.Join( ", ", t.GetGenericArguments().Select( GetTypeName ) )}>"
: t.Name; // Basic handling, needs improvement for full names/namespaces
}

2
Id.cs → lib/Id.cs
View File

@ -23,7 +23,7 @@ namespace lib
// TODO PERF Make span versions of all these functions
unsafe public static Id<T> Generate()
public static Id<T> Generate()
{
var buf = new byte[8];

166
lib/SerializableDictionary.cs_bad
View File

@ -1,166 +0,0 @@
using System;
using System.Collections.Generic;
using System.Text;
using System.Xml.Serialization;
using System.Xml;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
using System.IO;
using System.Security.Permissions;
namespace lib
{
[Serializable]
public class SerializableDictionary<TKey, TVal>: Dictionary<TKey, TVal>, IXmlSerializable, ISerializable
{
#region Constants
private const string DictionaryNodeName = "Dictionary";
private const string ItemNodeName = "Item";
private const string KeyNodeName = "Key";
private const string ValueNodeName = "Value";
#endregion
#region Constructors
public SerializableDictionary()
{
}
public SerializableDictionary( IDictionary<TKey, TVal> dictionary )
: base( dictionary )
{
}
public SerializableDictionary( IEqualityComparer<TKey> comparer )
: base( comparer )
{
}
public SerializableDictionary( int capacity )
: base( capacity )
{
}
public SerializableDictionary( IDictionary<TKey, TVal> dictionary, IEqualityComparer<TKey> comparer )
: base( dictionary, comparer )
{
}
public SerializableDictionary( int capacity, IEqualityComparer<TKey> comparer )
: base( capacity, comparer )
{
}
#endregion
#region ISerializable Members
protected SerializableDictionary( SerializationInfo info, StreamingContext context )
{
int itemCount = info.GetInt32("ItemCount");
for( int i = 0; i < itemCount; i++ )
{
KeyValuePair<TKey, TVal> kvp = (KeyValuePair<TKey, TVal>)info.GetValue(String.Format( $"Item{i}" ), typeof(KeyValuePair<TKey, TVal>));
this.Add( kvp.Key, kvp.Value );
}
}
//[SecurityPermission( SecurityAction.LinkDemand, Flags = SecurityPermissionFlag.SerializationFormatter )]
void ISerializable.GetObjectData( SerializationInfo info, StreamingContext context )
{
info.AddValue( "ItemCount", this.Count );
int itemIdx = 0;
foreach( KeyValuePair<TKey, TVal> kvp in this )
{
info.AddValue( String.Format( $"Item{itemIdx}" ), kvp, typeof( KeyValuePair<TKey, TVal> ) );
itemIdx++;
}
}
#endregion
#region IXmlSerializable Members
void IXmlSerializable.WriteXml( System.Xml.XmlWriter writer )
{
//writer.WriteStartElement(DictionaryNodeName);
foreach( KeyValuePair<TKey, TVal> kvp in this )
{
writer.WriteStartElement( ItemNodeName );
writer.WriteStartElement( KeyNodeName );
KeySerializer.Serialize( writer, kvp.Key );
writer.WriteEndElement();
writer.WriteStartElement( ValueNodeName );
ValueSerializer.Serialize( writer, kvp.Value );
writer.WriteEndElement();
writer.WriteEndElement();
}
//writer.WriteEndElement();
}
void IXmlSerializable.ReadXml( System.Xml.XmlReader reader )
{
if( reader.IsEmptyElement )
{
return;
}
// Move past container
if( !reader.Read() )
{
throw new XmlException( "Error in Deserialization of Dictionary" );
}
//reader.ReadStartElement(DictionaryNodeName);
while( reader.NodeType != XmlNodeType.EndElement )
{
reader.ReadStartElement( ItemNodeName );
reader.ReadStartElement( KeyNodeName );
TKey key = (TKey)KeySerializer.Deserialize(reader);
reader.ReadEndElement();
reader.ReadStartElement( ValueNodeName );
TVal value = (TVal)ValueSerializer.Deserialize(reader);
reader.ReadEndElement();
reader.ReadEndElement();
this.Add( key, value );
reader.MoveToContent();
}
//reader.ReadEndElement();
reader.ReadEndElement(); // Read End Element to close Read of containing node
}
System.Xml.Schema.XmlSchema IXmlSerializable.GetSchema()
{
return null;
}
#endregion
#region Private Properties
protected XmlSerializer ValueSerializer
{
get
{
if( valueSerializer == null )
{
valueSerializer = new XmlSerializer( typeof( TVal ) );
}
return valueSerializer;
}
}
private XmlSerializer KeySerializer
{
get
{
if( keySerializer == null )
{
keySerializer = new XmlSerializer( typeof( TKey ) );
}
return keySerializer;
}
}
#endregion
#region Private Members
private XmlSerializer keySerializer = null;
private XmlSerializer valueSerializer = null;
#endregion
}
}

677
lib/VersionFormatter.cs_bad
View File

@ -1,677 +0,0 @@
using System;
using System.IO;
using System.Reflection;
using System.Collections;
using System.Diagnostics;
//using System.Globalization;
//using System.ComponentModel;
using System.Runtime.Serialization;
namespace lib
{
/// <summary>
///
/// </summary>
public class VersionFormatter: IFormatter
{
public enum ETypes
{
Array,
Int32,
Ref,
Object,
EndObject,
Single,
Double,
Char,
String,
Boolean,
EndStream,
}
public VersionFormatter()
{
//
// TODO: Add constructor logic here
//
}
#region Useless
public ISurrogateSelector SurrogateSelector
{
get
{
return null;
}
set
{
}
}
public SerializationBinder Binder
{
get
{
return null;
}
set
{
}
}
public StreamingContext Context
{
get
{
return new StreamingContext();
}
set
{
}
}
#endregion Useless
Queue m_objectsToBeDeserialized = new Queue();
Hashtable m_alreadyDeserialzied = new Hashtable();
//int m_GUID = 0;
#region Serialize
public void Serialize( Stream stream, object obj )
{
//Default is 4k
//BufferedStream bufStream = new BufferedStream( stream );
BinaryWriter writer = new BinaryWriter( stream );
writeObject( writer, obj );
while( m_objectsToBeDeserialized.Count != 0 )
{
object objToDes = m_objectsToBeDeserialized.Dequeue();
writeObject( writer, objToDes );
}
writer.Write( (char)ETypes.EndStream );
}
void writeRefAndSched( BinaryWriter writer, object obj )
{
//if( m_alreadyDeserialzied[ obj.GetType().GetArrayRank(
if( obj == null )
{
writer.Write( 0 );
return;
}
//Now write the address.
//Bad bad. Need to do this correctly.
int objRef = obj.GetHashCode();
writer.Write( objRef );
if( m_alreadyDeserialzied[obj] == null )
{
m_alreadyDeserialzied[obj] = obj;
m_objectsToBeDeserialized.Enqueue( obj );
}
}
void dispatchWrite( BinaryWriter writer, object parentObj, FieldInfo fi )
{
string typeName = fi.FieldType.Name;
string name = fi.Name;
if( fi.IsNotSerialized )
{
return;
}
if( fi.FieldType.IsArray )
{
writer.Write( (char)ETypes.Array );
writer.Write( name.GetHashCode() );
writeArray( writer, (Array)fi.GetValue( parentObj ) );
}
else if( ( fi.FieldType.IsClass || fi.FieldType.IsInterface ) && typeName != "String" )
{
writer.Write( (char)ETypes.Ref );
writer.Write( name.GetHashCode() );
writeRefAndSched( writer, fi.GetValue( parentObj ) );
}
else if( fi.FieldType.IsEnum )
{
writer.Write( (char)ETypes.Int32 );
writer.Write( name.GetHashCode() );
write( writer, Convert.ToInt32( fi.GetValue( parentObj ) ) );
}
else
{
switch( typeName )
{
case "Int32":
writer.Write( (char)ETypes.Int32 );
writer.Write( name.GetHashCode() );
write( writer, Convert.ToInt32( fi.GetValue( parentObj ) ) );
break;
case "Single":
writer.Write( (char)ETypes.Single );
writer.Write( name.GetHashCode() );
write( writer, Convert.ToSingle( fi.GetValue( parentObj ) ) );
break;
case "Double":
writer.Write( (char)ETypes.Double );
writer.Write( name.GetHashCode() );
write( writer, Convert.ToDouble( fi.GetValue( parentObj ) ) );
break;
case "Char":
writer.Write( (char)ETypes.Char );
writer.Write( name.GetHashCode() );
write( writer, Convert.ToChar( fi.GetValue( parentObj ) ) );
break;
case "String":
writer.Write( (char)ETypes.String );
writer.Write( name.GetHashCode() );
write( writer, Convert.ToString( fi.GetValue( parentObj ) ) );
break;
case "Boolean":
writer.Write( (char)ETypes.Boolean );
writer.Write( name.GetHashCode() );
writer.Write( Convert.ToBoolean( fi.GetValue( parentObj ) ) );
break;
default:
Console.WriteLine( "VersionFormatter does not understand type " + typeName );
break;
}
}
}
void writeArray( BinaryWriter writer, Array array )
{
if( array == null )
{
writer.Write( (int)-1 );
return;
}
writer.Write( array.Length );
foreach( object obj in array )
{
writeRefAndSched( writer, obj );
}
}
void getAllFields( object obj, ArrayList list )
{
Type t = obj.GetType();
while( t != null )
{
FieldInfo[] fiArr = t.GetFields( BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.DeclaredOnly );
list.AddRange( fiArr );
t = t.BaseType;
}
}
void writeObject( BinaryWriter writer, object obj )
{
Type objType = obj.GetType();
writer.Write( (char)ETypes.Object );
writer.Write( objType.FullName );
int objRef = obj.GetHashCode();
writer.Write( objRef );
ArrayList list = new ArrayList();
getAllFields( obj, list );
foreach( FieldInfo fi in list )
{
dispatchWrite( writer, obj, fi );
}
writer.Write( (char)ETypes.EndObject );
}
void write<TType>( BinaryWriter wr, TType val )
{
//wr.Write( val );
}
/*
void writeInt( BinaryWriter writer, int val )
{
writer.Write( val );
}
void writeSingle( BinaryWriter writer, float val )
{
writer.Write( val );
}
void writeDouble( BinaryWriter writer, double val )
{
writer.Write( val );
}
void writeChar( BinaryWriter writer, char val )
{
writer.Write( val );
}
void writeString( BinaryWriter writer, string val )
{
writer.Write( val );
}
void writeBool( BinaryWriter writer, bool val )
{
writer.Write( val );
}
*/
#endregion Serialize
#region Deserialize
class Fixup
{
public Fixup( int guid, object obj, FieldInfo fi )
{
m_guid = guid;
m_obj = obj;
m_fi = fi;
}
public Fixup( int guid, object obj, int index )
{
m_guid = guid;
m_obj = obj;
m_index = index;
}
public readonly int m_guid = 0;
public readonly object m_obj = null;
public readonly FieldInfo m_fi = null;
public readonly int m_index= -1;
}
Hashtable m_mapGUIDToObject = new Hashtable();
ArrayList m_fixupList = new ArrayList();
ArrayList m_desObjects = new ArrayList();
public object Deserialize( Stream stream )
{
BinaryReader reader = new BinaryReader( stream );
object objRoot = null;
//Read in the first object.
{
ETypes type = (ETypes)reader.ReadChar();
Debug.Assert( type == ETypes.Object );
objRoot = readObject( reader );
m_desObjects.Add( objRoot );
}
bool readObjects = true;
while( readObjects )
{
ETypes type = (ETypes)reader.ReadChar();
Debug.Assert( type == ETypes.Object || type == ETypes.EndStream );
if( type == ETypes.Object )
{
object obj = readObject( reader );
m_desObjects.Add( obj );
}
else
{
Debug.Assert( type == ETypes.EndStream );
readObjects = false;
}
}
foreach( Fixup fu in m_fixupList )
{
//Fixup fix = m_fixups[
object obj = m_mapGUIDToObject[ fu.m_guid ];
if( obj != null )
{
if( fu.m_fi != null )
{
fu.m_fi.SetValue( fu.m_obj, obj );
}
else
{
Debug.Assert( fu.m_index >= 0 );
object []array = (object [])fu.m_obj;
array[fu.m_index] = obj;
}
}
else
{
Console.WriteLine( "Obj to ref is null." );
}
}
foreach( object obj in m_desObjects )
{
if( typeof( IDeserializationCallback ).IsAssignableFrom( obj.GetType() ) )
{
IDeserializationCallback desCB = (IDeserializationCallback)obj;
if( desCB != null )
{
desCB.OnDeserialization( this );
}
}
}
return objRoot;
}
bool dispatchRead( BinaryReader reader, object obj, Hashtable ht )
{
//Read the type
ETypes type = (ETypes)reader.ReadChar();
if( type == ETypes.EndObject )
{
return false;
}
int nameHash = reader.ReadInt32();
FieldInfo fi = (FieldInfo)ht[ nameHash ];
if( fi == null )
{
Console.WriteLine( "Field no longer exists" );
}
try
{
switch( type )
{
case ETypes.Array:
readArray( reader, obj, fi );
break;
case ETypes.Int32:
readInt( reader, obj, fi );
break;
case ETypes.Single:
readSingle( reader, obj, fi );
break;
case ETypes.Double:
readDouble( reader, obj, fi );
break;
case ETypes.Char:
readChar( reader, obj, fi );
break;
case ETypes.Boolean:
readBool( reader, obj, fi );
break;
case ETypes.String:
readString( reader, obj, fi );
break;
case ETypes.Ref:
readRef( reader, obj, fi );
break;
case ETypes.Object:
readObject( reader );
break;
default:
Debug.Fail( "Unknown type on read." );
break;
}
}
catch( Exception ex )
{
Console.WriteLine( "Exception: " + ex.Message );
Console.WriteLine( "Stack: " + ex.StackTrace );
}
return true;
}
object createObject( string objTypeName )
{
Assembly[] ass = AppDomain.CurrentDomain.GetAssemblies();
foreach( Assembly a in ass )
{
Type t = a.GetType( objTypeName );
if( t != null )
{
object obj = FormatterServices.GetUninitializedObject( t );
if( obj != null )
{
return obj;
}
}
}
return null;
}
object readObject( BinaryReader reader )
{
//ETypes type = (ETypes)reader.ReadChar();
//Debug.Assert( type == ETypes.Object, "Expecting type Object" );
string objTypeName = reader.ReadString();
int objGUID = reader.ReadInt32();
try
{
object obj = createObject( objTypeName );
m_mapGUIDToObject[objGUID] = obj;
ArrayList list = new ArrayList();
Hashtable ht = new Hashtable();
if( obj != null )
{
getAllFields( obj, list );
foreach( FieldInfo fi in list )
{
ht[fi.Name.GetHashCode()] = fi;
}
}
while( dispatchRead( reader, obj, ht ) )
{
}
return obj;
}
catch( Exception ex )
{
Console.WriteLine( "Exception: " + ex.Message );
}
return null;
}
void readArray( BinaryReader reader, object obj, FieldInfo fi )
{
int length = reader.ReadInt32();
if( length < 0 )
{
if( fi == null )
return;
fi.SetValue( obj, null );
return;
}
object[] array = new object[length];
if( fi != null )
{
fi.SetValue( obj, array );
}
for( int i = 0; i < length; ++i )
{
int val = reader.ReadInt32();
//m_fixups[ val ] = new Fixup( obj, fi );
if( fi != null )
{
m_fixupList.Add( new Fixup( val, array, i ) );
}
}
}
void readRef( BinaryReader reader, object obj, FieldInfo fi )
{
int val = reader.ReadInt32();
//m_fixups[ val ] = new Fixup( obj, fi );
m_fixupList.Add( new Fixup( val, obj, fi ) );
}
void readInt( BinaryReader reader, object obj, FieldInfo fi )
{
int val = reader.ReadInt32();
if( fi == null )
return;
if( !fi.FieldType.IsEnum )
{
fi.SetValue( obj, val );
}
else
{
object enumVal = Enum.Parse( fi.FieldType, val.ToString() );
fi.SetValue( obj, Convert.ChangeType( enumVal, fi.FieldType ) );
}
}
void readSingle( BinaryReader reader, object obj, FieldInfo fi )
{
float val = reader.ReadSingle();
if( fi == null )
return;
fi.SetValue( obj, val );
}
void readDouble( BinaryReader reader, object obj, FieldInfo fi )
{
double val = reader.ReadDouble();
if( fi == null )
return;
fi.SetValue( obj, val );
}
void readChar( BinaryReader reader, object obj, FieldInfo fi )
{
char val = reader.ReadChar();
if( fi == null )
return;
fi.SetValue( obj, val );
}
void readString( BinaryReader reader, object obj, FieldInfo fi )
{
string val = reader.ReadString();
if( fi == null )
return;
fi.SetValue( obj, val );
}
void readBool( BinaryReader reader, object obj, FieldInfo fi )
{
bool val = reader.ReadBoolean();
if( fi == null )
return;
fi.SetValue( obj, val );
}
#endregion Deserialize
}
}

1094
lib/XmlFormatter.cs_bad
View File

File diff suppressed because it is too large Load Diff

2
lib/XmlFormatter2.cs
View File

@ -92,7 +92,7 @@ namespace lib
public ser.Types TypesDefault = ser.Types.Fields;
}
public class XmlFormatter2 : IFormatter
public class XmlFormatter2
{

45
logging/Log.cs
View File

@ -9,7 +9,6 @@ using System.Reflection;
using System.Collections.Concurrent;
using System.Linq;
using System.Threading;
using lib.Net;
using System.Text;
//using System.Threading.Tasks;
@ -141,14 +140,14 @@ static public class log
// It would be better to use a ManualResetEvent here if waiting is truly needed.
Thread.Sleep( 500 ); // Simple wait
var processId = Process.GetCurrentProcess().Id;
LogGC.PrintRuntimeGCEvents( processId );
//LogGC.PrintRuntimeGCEvents( processId );
}
static void StartTracing()
{
// See above comment
Thread.Sleep( 500 );
Tracing.TraceLogMonitor.Run();
//Tracing.TraceLogMonitor.Run();
}
#endregion // CLR Logging
@ -626,34 +625,56 @@ static public class log
logBase( $"{header}{dbgExpObj}[{type.Name}] ->", level, dbgPath, dbgLine, dbgMethod, cat, dbgExpObj );
// 6. Handle Complex Objects (Properties)
if( maxDepth == 0 )
if( maxDepth < 0 )
{
return;
}
List<MemberInfo> members = new();
refl.GetAllMembers( type, members );
// Get all readable properties
var props = type.GetProperties( BindingFlags.Public | BindingFlags.Instance )
.Where( p => p.CanRead && p.GetIndexParameters().Length == 0 );
//var props = type.GetProperties( BindingFlags.Public | BindingFlags.Private | BindingFlags.Instance )
// .Where( p => p.CanRead && p.GetIndexParameters().Length == 0 );
var props = members.Where( ( mi) => true );
var nextHeader = $"{header}{header}";
//bool firstProp = true;
foreach( var pi in props )
foreach( var mi in members )
{
var name = pi.Name;
var piType = pi.PropertyType;
var is_pi = mi is PropertyInfo pi;
var is_fi = mi is FieldInfo fi;
var name = mi.Name;
var piType = mi.MemberType;
try
{
var value = pi.GetValue( obj );
object? value = GetValue( obj, mi );
props_r( value, nextHeader, --maxDepth, level, cat, prefix, dbgPath, dbgLine, dbgMethod, name );
}
catch( Exception ex )
{
logBase( $"{nextHeader}{name}[{piType.Name}] ex {ex.Message}", level, dbgPath, dbgLine, dbgMethod, cat, name );
logBase( $"{nextHeader}{name}[{mi.Name}] ex {ex.Message}", level, dbgPath, dbgLine, dbgMethod, cat, name );
}
}
}
static object GetValue( object obj, MemberInfo mi )
{
if( mi is PropertyInfo pi )
{
return pi.GetValue( obj );
}
if( mi is FieldInfo fi )
{
return fi.GetValue( obj );
}
return null;
}
//This might seem a little odd, but the intent is that usually you wont need to set notExpectedValue.
static public void expected<T>( T value, string falseString, string trueString = "", T? notExpectedValue = default( T ) )
@ -825,7 +846,7 @@ static public class log
StartThread();
LogGC.RegisterObjectId( s_lock );
//LogGC.RegisterObjectId( s_lock );
info( $"startup END", cat: "log.startup" );
}

3
math/BoundingBoxExt.cs
View File

@ -87,6 +87,7 @@ namespace math
Vec3.TransformCoordinate( ref center, ref world, out Center );
#if UNSAFE
// Update world matrix into absolute form
unsafe
{
@ -99,7 +100,7 @@ namespace math
++matrixData;
}
}
#endif
Vec3.TransformNormal( ref extent, ref world, out Extent );
}

4
math/BoundingSphere.cs
View File

@ -185,6 +185,7 @@ namespace math
return CollisionHelper.SphereContainsSphere( ref this, ref sphere );
}
#if UNSAFE
/// <summary>
/// Constructs a <see cref="math.BoundingSphere"/> that fully contains the given points.
/// </summary>
@ -199,7 +200,6 @@ namespace math
FromPoints( (IntPtr)pointsPtr, 0, points.Length, lib.Util.SizeOf<Vec3>(), out result );
}
}
/// <summary>
/// Constructs a <see cref="math.BoundingSphere" /> that fully contains the given unmanaged points.
/// </summary>
@ -251,7 +251,6 @@ namespace math
result.Center = center;
result.Radius = radius;
}
/// <summary>
/// Constructs a <see cref="math.BoundingSphere"/> that fully contains the given points.
/// </summary>
@ -263,6 +262,7 @@ namespace math
FromPoints( points, out result );
return result;
}
#endif
/// <summary>
/// Constructs a <see cref="math.BoundingSphere"/> from a given box.

28
math/CollisionHelper.cs
View File

@ -1474,13 +1474,23 @@ namespace math
return ContainmentType.Contains;
}
/// <summary>
/// Determines whether a <see cref="BoundingFrustum" /> intersects or contains an AABB determined by its center and extent.
/// Faster variant specific for frustum culling.
/// </summary>
/// <param name="frustum">The frustum.</param>
/// <param name="boundingBoxExt">The bounding box ext.</param>
/// <returns><c>true</c> if XXXX, <c>false</c> otherwise.</returns>
/// <summary>
/// Determines whether a <see cref="BoundingFrustum" /> intersects or contains an AABB determined by its center and extent.
/// Faster variant specific for frustum culling.
/// </summary>
/// <param name="frustum">The frustum.</param>
/// <param name="boundingBoxExt">The bounding box ext.</param>
/// <returns><c>true</c> if XXXX, <c>false</c> otherwise.</returns>
public static bool FrustumContainsBox( ref BoundingFrustum frustum, ref BoundingBoxExt boundingBoxExt )
{
throw new NotImplementedException();
throw new NotImplementedException();
return false;
}
#if UNSAFE
public static bool FrustumContainsBox( ref BoundingFrustum frustum, ref BoundingBoxExt boundingBoxExt )
{
unsafe
@ -1547,5 +1557,7 @@ namespace math
}
*/
}
}
#endif
}
}

2
math/Double2.cs
View File

@ -18,10 +18,12 @@ namespace math
[StructLayout( LayoutKind.Sequential, Pack = 4 )]
public struct Double2 : IEquatable<Double2>, IFormattable
{
#if UNSAFE
/// <summary>
/// The size of the <see cref="math.Double2"/> type, in bytes.
/// </summary>
public static readonly int SizeInBytes = lib.Util.SizeOf<Double2>();
#endif
/// <summary>
/// A <see cref="math.Double2"/> with all of its components set to zero.

14
math/Double3.cs
View File

@ -17,16 +17,20 @@ namespace math
[DataStyle( DataStyle.Compact )]
[StructLayout( LayoutKind.Sequential, Pack = 4 )]
public struct Double3 : IEquatable<Double3>, IFormattable
{
{
#if UNSAFE
/// <summary>
/// The size of the <see cref="math.Double3"/> type, in bytes.
/// </summary>
public static readonly int SizeInBytes = lib.Util.SizeOf<Double3>();
#endif // UNSAFE
/// <summary>
/// A <see cref="math.Double3"/> with all of its components set to zero.
/// </summary>
public static readonly Double3 Zero = new Double3();
/// <summary>
/// A <see cref="math.Double3"/> with all of its components set to zero.
/// </summary>
public static readonly Double3 Zero = new Double3();
/// <summary>
/// The X unit <see cref="math.Double3"/> (1, 0, 0).

10
math/Double4.cs
View File

@ -18,15 +18,17 @@ namespace math
[StructLayout( LayoutKind.Sequential, Pack = 4 )]
public struct Double4 : IEquatable<Double4>, IFormattable
{
#if UNSAFE
/// <summary>
/// The size of the <see cref="math.Double4"/> type, in bytes.
/// </summary>
public static readonly int SizeInBytes = lib.Util.SizeOf<Double4>();
#endif // UNSAFE
/// <summary>
/// A <see cref="math.Double4"/> with all of its components set to zero.
/// </summary>
public static readonly Double4 Zero = new Double4();
/// <summary>
/// A <see cref="math.Double4"/> with all of its components set to zero.
/// </summary>
public static readonly Double4 Zero = new Double4();
/// <summary>
/// The X unit <see cref="math.Double4"/> (1, 0, 0, 0).

11
math/Int2.cs
View File

@ -41,15 +41,18 @@ namespace math
[StructLayout( LayoutKind.Sequential, Pack = 4 )]
public struct Int2 : IEquatable<Int2>, IFormattable
{
#if UNSAFE
/// <summary>
/// The size of the <see cref="math.Int2"/> type, in bytes.
/// </summary>
public static readonly int SizeInBytes = lib.Util.SizeOf<Int2>();
#endif // UNSAFE
/// <summary>
/// A <see cref="math.Int2"/> with all of its components set to zero.
/// </summary>
public static readonly Int2 Zero = new Int2();
/// <summary>
/// A <see cref="math.Int2"/> with all of its components set to zero.
/// </summary>
public static readonly Int2 Zero = new Int2();
/// <summary>
/// The X unit <see cref="math.Int2"/> (1, 0, 0).

3
math/Int3.cs
View File

@ -41,10 +41,13 @@ namespace math
[StructLayout( LayoutKind.Sequential, Pack = 4 )]
public struct Int3 : IEquatable<Int3>, IFormattable
{
#if UNSAFE
/// <summary>
/// The size of the <see cref="Int3"/> type, in bytes.
/// </summary>
public static readonly int SizeInBytes = lib.Util.SizeOf<Int3>();
#endif // UNSAFE
/// <summary>
/// A <see cref="Int3"/> with all of its components set to zero.

11
math/Int4.cs
View File

@ -35,15 +35,18 @@ namespace math
[StructLayout( LayoutKind.Sequential, Pack = 4 )]
public struct Int4 : IEquatable<Int4>, IFormattable
{
#if UNSAFE
/// <summary>
/// The size of the <see cref = "Int4" /> type, in bytes.
/// </summary>
public static readonly int SizeInBytes = lib.Util.SizeOf<Int4>();
#endif // UNSAFE
/// <summary>
/// A <see cref = "Int4" /> with all of its components set to zero.
/// </summary>
public static readonly Int4 Zero = new Int4();
/// <summary>
/// A <see cref = "Int4" /> with all of its components set to zero.
/// </summary>
public static readonly Int4 Zero = new Int4();
/// <summary>
/// The X unit <see cref = "Int4" /> (1, 0, 0, 0).

10
math/MathUtil.cs
View File

@ -79,6 +79,14 @@ namespace math
/// The code is using the technique described by Bruce Dawson in
/// <a href="http://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/">Comparing Floating point numbers 2012 edition</a>.
/// </remarks>
public static bool NearEqual( float a, float b )
{
return Math.Abs( a - b ) < 0.00001f;
}
#if UNSAFE
public static unsafe bool NearEqual( float a, float b )
{
// Check if the numbers are really close -- needed
@ -102,7 +110,7 @@ namespace math
const int maxUlp = 4;
return ( ulp <= maxUlp );
}
#endif
/// <summary>
/// Determines whether the specified value is close to zero (0.0f).
/// </summary>

14
math/Matrix.cs
View File

@ -44,15 +44,18 @@ namespace math
[StructLayout( LayoutKind.Sequential, Pack = 4 )]
public struct Matrix : IEquatable<Matrix>, IFormattable
{
#if UNSAFE
/// <summary>
/// The size of the <see cref="math.Matrix"/> type, in bytes.
/// </summary>
public static readonly int SizeInBytes = lib.Util.SizeOf<Matrix>();
#endif // UNSAFE
/// <summary>
/// A <see cref="math.Matrix"/> with all of its components set to zero.
/// </summary>
public static readonly Matrix Zero = new Matrix();
/// <summary>
/// A <see cref="math.Matrix"/> with all of its components set to zero.
/// </summary>
public static readonly Matrix Zero = new Matrix();
/// <summary>
/// The identity <see cref="math.Matrix"/>.
@ -3168,6 +3171,7 @@ namespace math
return result;
}
#if UNSAFE
/// <summary>
/// Copies a nxm matrix to this instance.
/// </summary>
@ -3214,7 +3218,7 @@ namespace math
}
}
}
#endif
/// <summary>
/// Adds two matricies.
/// </summary>

11
math/Quaternion.cs
View File

@ -41,15 +41,18 @@ namespace math
[StructLayout( LayoutKind.Sequential, Pack = 4 )]
public struct Quaternion : IEquatable<Quaternion>, IFormattable
{
#if UNSAFE
/// <summary>
/// The size of the <see cref="math.Quaternion"/> type, in bytes.
/// </summary>
public static readonly int SizeInBytes = lib.Util.SizeOf<Quaternion>();
#endif // UNSAFE
/// <summary>
/// A <see cref="math.Quaternion"/> with all of its components set to zero.
/// </summary>
public static readonly Quaternion Zero = new Quaternion();
/// <summary>
/// A <see cref="math.Quaternion"/> with all of its components set to zero.
/// </summary>
public static readonly Quaternion Zero = new Quaternion();
/// <summary>
/// A <see cref="math.Quaternion"/> with all of its components set to one.

3
math/UInt4.cs
View File

@ -34,10 +34,13 @@ namespace math
[StructLayout( LayoutKind.Sequential, Pack = 4 )]
public struct UInt4 : IEquatable<UInt4>, IFormattable
{
#if UNSAFE
/// <summary>
/// The size of the <see cref = "UInt4" /> type, in bytes.
/// </summary>
public static readonly int SizeInBytes = lib.Util.SizeOf<UInt4>();
#endif // UNSAFE
/// <summary>
/// A <see cref = "UInt4" /> with all of its components set to zero.

3
math/Vector2.cs
View File

@ -42,11 +42,12 @@ namespace math
[StructLayout( LayoutKind.Sequential, Pack = 4 )]
public struct Vec2 : IEquatable<Vec2>, IFormattable
{
#if UNSAFE
/// <summary>
/// The size of the <see cref="math.Vec2"/> type, in bytes.
/// </summary>
public static readonly int SizeInBytes = lib.Util.SizeOf<Vec2>();
#endif
/// <summary>
/// A <see cref="math.Vec2"/> with all of its components set to zero.
/// </summary>

3
math/Vector3.cs
View File

@ -43,11 +43,12 @@ namespace math
[StructLayout( LayoutKind.Sequential, Pack = 4 )]
public struct Vec3 : IEquatable<Vec3>, IFormattable
{
#if UNSAFE
/// <summary>
/// The size of the <see cref="math.Vec3"/> type, in bytes.
/// </summary>
public static readonly int SizeInBytes = lib.Util.SizeOf<Vec3>();
#endif
/// <summary>
/// A <see cref="math.Vec3"/> with all of its components set to zero.
/// </summary>

3
math/Vector4.cs
View File

@ -42,11 +42,12 @@ namespace math
[StructLayout( LayoutKind.Sequential, Pack = 4 )]
public struct Vec4 : IEquatable<Vec4>, IFormattable
{
#if UNSAFE
/// <summary>
/// The size of the <see cref="math.Vec4"/> type, in bytes.
/// </summary>
public static readonly int SizeInBytes = lib.Util.SizeOf<Vec4>();
#endif
/// <summary>
/// A <see cref="math.Vec4"/> with all of its components set to zero.
/// </summary>

1
math/fn.cs
View File

@ -1,4 +1,3 @@
using Microsoft.CodeAnalysis.CSharp.Syntax;
using System;
using System.ComponentModel;

87
mod/Modules.cs
View File

@ -1,87 +0,0 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
//using System.Threading.Tasks;
using System.Diagnostics;
using System.Reflection;
namespace mod
{
[Serializable]
public class Config : lib.Config
{
public String name = "Generic";
}
public class View
{
}
public class Base
{
public Config Cfg { get { return m_cfg; } }
public Base( Config cfg )
{
m_cfg = cfg;
}
private Config m_cfg;
}
[Serializable]
public class FluidConfig : Config
{
public String type = "none";
}
public class FluidBase : Base
{
public new FluidConfig Cfg { get { return (FluidConfig)base.Cfg; } }
public FluidBase( FluidConfig cfg )
: base( cfg )
{
}
}
[Serializable]
public class SystemConfig : Config
{
public String type = "none";
}
public class System
{
public SystemConfig Cfg { get { return m_cfg; } }
public System( SystemConfig cfg )
{
m_cfg = cfg;
}
private SystemConfig m_cfg;
}
}

198
net/Conn.cs
View File

@ -1,198 +0,0 @@
#nullable enable
using System;
using System.Runtime.Serialization;
using System.Net.Sockets;
using System.IO;
using System.Diagnostics.CodeAnalysis;
//using Util;
namespace lib
{
public interface IFormatter
{
//
// Summary:
// Gets or sets the System.Runtime.Serialization.SerializationBinder that performs
// type lookups during deserialization.
//
// Returns:
// The System.Runtime.Serialization.SerializationBinder that performs type lookups
// during deserialization.
SerializationBinder? Binder { get; set; }
//
// Summary:
// Gets or sets the System.Runtime.Serialization.StreamingContext used for serialization
// and deserialization.
//
// Returns:
// The System.Runtime.Serialization.StreamingContext used for serialization and
// deserialization.
StreamingContext Context { get; set; }
//
// Summary:
// Gets or sets the System.Runtime.Serialization.SurrogateSelector used by the current
// formatter.
//
// Returns:
// The System.Runtime.Serialization.SurrogateSelector used by this formatter.
ISurrogateSelector? SurrogateSelector { get; set; }
//
// Summary:
// Deserializes the data on the provided stream and reconstitutes the graph of objects.
//
//
// Parameters:
// serializationStream:
// The stream that contains the data to deserialize.
//
// Returns:
// The top object of the deserialized graph.
[RequiresDynamicCode( "BinaryFormatter serialization uses dynamic code generation, the type of objects being processed cannot be statically discovered." )]
[RequiresUnreferencedCode( "BinaryFormatter serialization is not trim compatible because the type of objects being processed cannot be statically discovered." )]
object Deserialize( Stream serializationStream );
//
// Summary:
// Serializes an object, or graph of objects with the given root to the provided
// stream.
//
// Parameters:
// serializationStream:
// The stream where the formatter puts the serialized data. This stream can reference
// a variety of backing stores (such as files, network, memory, and so on).
//
// graph:
// The object, or root of the object graph, to serialize. All child objects of this
// root object are automatically serialized.
[RequiresUnreferencedCode( "BinaryFormatter serialization is not trim compatible because the type of objects being processed cannot be statically discovered." )]
void Serialize( Stream serializationStream, object graph );
}
public interface IProcess
{
void process( object obj );
}
public interface ISerDes<T> where T : IFormatter
{
T getInstance();
}
public class NewEveryCall<T> : ISerDes<T> where T : IFormatter, new()
{
public T getInstance()
{
return new T();
}
}
public class Conn
{
public static int BufferSize = 2048;
}
public class Conn<T, TInst> : Conn
where T : IFormatter, new()
where TInst : ISerDes<T>, new()
{
public Socket Sock { get { return m_socket; } }
public Stream Stream { get { return m_streamNet; } }
private TInst m_formatter = new TInst();
public Conn( Socket sock, IProcess proc )
{
m_socket = sock;
sock.NoDelay = true;
m_streamNet = new NetworkStream( m_socket );
m_proc = proc;
}
public object receiveObject()
{
return receiveObject( Stream );
}
public object receiveObject( Stream stream )
{
object obj = new object();
var formatter = m_formatter.getInstance();
try
{
obj = formatter.Deserialize( stream );
}
catch( System.Xml.XmlException ex )
{
log.error( $"Outer Exception {ex.Message}" );
}
return obj;
}
public void send( object obj )
{
var formatter = m_formatter.getInstance();
try
{
var ms = new MemoryStream( BufferSize );
formatter.Serialize( ms, obj );
//var str = System.Text.Encoding.Default.GetString( mm_buffer, 0, (int)ms.Position );
//log.info( $"Sent data {str} of length {ms.Position}" );
//log.info( $"Sent {obj}" );
byte[] byteSize = BitConverter.GetBytes( (uint)ms.Position );
m_streamNet.Write( byteSize, 0, 4 );
m_streamNet.Write( ms.GetBuffer(), 0, (int)ms.Position );
m_streamNet.Flush();
}
catch( Exception e )
{
log.warn( $"Exception sending obj {obj} of {e}" );
throw;
}
}
public virtual void receive( object obj )
{
if( m_proc != null )
m_proc.process( obj );
}
Socket m_socket;
NetworkStream m_streamNet;
IProcess m_proc;
//private BufferedStream m_streamBufIn;
//private BufferedStream m_streamBufOut;
}
}

66
net/FSM.cs
View File

@ -1,66 +0,0 @@

using System;
namespace net;
public class Context
{
}
public class State<T, CTX>
where T : State<T, CTX>
where CTX : Context
{
virtual public void onEnter( CTX ctx, State<T, CTX> oldState )
{
}
virtual public void onExit( CTX ctx, State<T, CTX> newState )
{
}
}
/*
public class PotentialState<T, CTX> : State<T, CTX>
where T : State<T, CTX>
where CTX : Context
{
virtual public void onEnter(CTX ctx, State<T, CTX> oldState)
{
}
virtual public void onExit(CTX ctx, State<T, CTX> newState)
{
}
}
*/
public class FSM<T, CTX, ST>
where T : FSM<T, CTX, ST>
where CTX : Context
where ST : State<ST, CTX>
{
public CTX Context { get; private set; }
public ST State { get; private set; }
public FSM( CTX context, ST state )
{
Context = context;
State = state;
}
public void Transition( ST newState )
{
}
}

103
net/NetMsg.cs
View File

@ -1,103 +0,0 @@
using System;
namespace lib.Net
{
[Serializable]
public class Msg
{
public Msg()
{
}
}
[Serializable]
public class Login
{
public Login( String name, String pass )
{
m_username = name;
m_password = pass;
}
public readonly String m_username;
public readonly String m_password;
}
[Serializable]
public class LoginResp
{
public LoginResp( bool resp )
{
m_resp = resp;
}
public readonly bool m_resp;
}
#region Admin Messages
//Subclasses of this need to be on an admin client.
[Serializable]
public class Admin
{
};
[Serializable]
public class CreateEntity : Admin
{
}
[Serializable]
public class MoveEntity : Admin
{
}
#endregion
[Serializable]
public class EntityBase
{
public EntityBase( int id )
{
m_id = id;
}
public readonly int m_id;
};
[Serializable]
public class EntityPos : EntityBase
{
public EntityPos( int id, float x, float y, float z ) :
base( id )
{
m_x = x;
m_y = y;
m_z = z;
}
public readonly float m_x;
public readonly float m_y;
public readonly float m_z;
}
[Serializable]
public class EntityDesc : EntityBase
{
public EntityDesc( int id ) :
base( id )
{
}
//Should an entity have a mesh? Be made up of multiple meshes?
public readonly String m_mesh;
}
}

47
prof/AddressStack.cs
View File

@ -1,47 +0,0 @@
using System.Collections.Generic;
namespace Tracing
{
public class AddressStack
{
// the first frame is the address of the last called method
private readonly List<ulong> _stack;
public AddressStack( int capacity )
{
_stack = new List<ulong>( capacity );
}
// No need to override GetHashCode because we don't want to use it as a key in a dictionary
public override bool Equals( object obj )
{
if( obj == null )
return false;
var stack = obj as AddressStack;
if( stack == null )
return false;
var frameCount = _stack.Count;
if( frameCount != stack._stack.Count )
return false;
for( int i = 0; i < frameCount; i++ )
{
if( _stack[i] != stack._stack[i] )
return false;
}
return true;
}
public override int GetHashCode() => _stack.GetHashCode();
public IReadOnlyList<ulong> Stack => _stack;
public void AddFrame( ulong address )
{
_stack.Add( address );
}
}
}

314
prof/Memory.cs
View File

@ -1,314 +0,0 @@
using Microsoft.Diagnostics.Tracing;
using Microsoft.Diagnostics.Tracing.Parsers;
using Microsoft.Diagnostics.Tracing.Parsers.Clr;
using Microsoft.Diagnostics.Tracing.Parsers.Kernel;
using Microsoft.Diagnostics.Tracing.Session;
using ProfilerHelpers;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;
namespace Tracing
{
public class Memory
{
private readonly TraceEventSession _session;
private readonly PerProcessProfilingState _processes;
// because we are not interested in self monitoring
private readonly int _currentPid;
private int _started = 0;
public Memory( TraceEventSession session, PerProcessProfilingState processes )
{
_session = session;
_processes = processes;
_currentPid = Process.GetCurrentProcess().Id;
}
public async Task StartAsync( bool allAllocations )
{
if( Interlocked.CompareExchange( ref _started, 1, 0 ) == 1 )
{
throw new InvalidOperationException( "Impossible to start profiling more than once." );
}
await Task.Factory.StartNew( () =>
{
using( _session )
{
log.info( $"SetupProviders" );
SetupProviders( _session, allAllocations );
log.info( $"SetupListeners" );
SetupListeners( _session.Source );
log.info( $"Source.Process()" );
while( _session.Source.Process() )
{
Task.Delay( 1 );
}
log.info( $"Done" );
}
} );
}
private void SetupProviders( TraceEventSession session, bool noSampling )
{
// Note: the kernel provider MUST be the first provider to be enabled
// If the kernel provider is not enabled, the callstacks for CLR events are still received
// but the symbols are not found (except for the application itself)
// Maybe a TraceEvent implementation details triggered when a module (image) is loaded
var success = true;
//*
log.info( $"EnableKernelProvider" );
success = log.var( session.EnableKernelProvider( KernelTraceEventParser.Keywords.None |
// KernelTraceEventParser.Keywords.ImageLoad |
// KernelTraceEventParser.Keywords.Process
0,
KernelTraceEventParser.Keywords.None
) );
log.info( $"EnableKernelProvider {success}" );
//*/
// The CLR source code indicates that the provider must be set before the monitored application starts
// Note: no real difference between High and Low
ClrTraceEventParser.Keywords eventsKeyword = noSampling
? ClrTraceEventParser.Keywords.GCSampledObjectAllocationLow | ClrTraceEventParser.Keywords.GCSampledObjectAllocationHigh
: ClrTraceEventParser.Keywords.GCSampledObjectAllocationLow
;
log.info( $"EnableProvider" );
success = log.var( session.EnableProvider(
ClrTraceEventParser.ProviderGuid,
TraceEventLevel.Verbose, // this is needed in order to receive GCSampledObjectAllocation event
(ulong)(
eventsKeyword |
// required to receive the BulkType events that allows
// mapping between the type ID received in the allocation events
ClrTraceEventParser.Keywords.GCHeapAndTypeNames |
ClrTraceEventParser.Keywords.Type |
// events related to JITed methods
ClrTraceEventParser.Keywords.Jit | // Turning on JIT events is necessary to resolve JIT compiled code
ClrTraceEventParser.Keywords.JittedMethodILToNativeMap | // This is needed if you want line number information in the stacks
ClrTraceEventParser.Keywords.Loader | // You must include loader events as well to resolve JIT compiled code.
// this is mandatory to get the callstacks in each CLR event payload.
//ClrTraceEventParser.Keywords.Stack |
0
)
) );
log.info( $"EnableProvider {success}" );
// Note: ClrRundown is not needed because only new processes will be monitored
}
private void SetupListeners( ETWTraceEventSource source )
{
// register for high and low keyword
// if both are set, each allocation will trigger an event (beware performance issues...)
source.Clr.GCSampledObjectAllocation += OnSampleObjectAllocation;
// required to receive the mapping between type ID (received in GCSampledObjectAllocation)
// and their name (received in TypeBulkType)
source.Clr.TypeBulkType += OnTypeBulkType;
// messages to get callstacks
// the correlation seems to be as "simple" as taking the last event on the same thread
source.Clr.ClrStackWalk += OnClrStackWalk;
// needed to get JITed method details
source.Clr.MethodLoadVerbose += OnMethodDetails;
source.Clr.MethodDCStartVerboseV2 += OnMethodDetails;
source.Clr.ContentionLockCreated += OnLockCreated;
source.Clr.ContentionStart += OnLockStart;
source.Clr.ContentionStop += OnLockStop;
// get notified when a module is load to map the corresponding symbols
source.Kernel.ImageLoad += OnImageLoad;
}
private void OnLockCreated( ContentionLockCreatedTraceData data )
{
}
private void OnLockStart( ContentionStartTraceData data )
{
}
private void OnLockStop( ContentionStopTraceData data )
{
}
private void OnImageLoad( ImageLoadTraceData data )
{
if( FilterOutEvent( data ) )
return;
GetProcessMethods( data.ProcessID ).AddModule( data.FileName, data.ImageBase, data.ImageSize );
log.info( $"{data.ProcessID}.{data.ThreadID} --> {data.FileName}" );
}
private void OnMethodDetails( MethodLoadUnloadVerboseTraceData data )
{
if( FilterOutEvent( data ) )
return;
// care only about jitted methods
if( !data.IsJitted )
return;
var method = GetProcessMethods( data.ProcessID )
.Add( data.MethodStartAddress, data.MethodSize, data.MethodNamespace, data.MethodName, data.MethodSignature );
log.info( $"0x{data.MethodStartAddress.ToString( "x12" )} - {data.MethodSize,6} | {data.MethodName}" );
}
private MethodStore GetProcessMethods( int pid )
{
if( !_processes.Methods.TryGetValue( pid, out var methods ) )
{
methods = new MethodStore( pid );
_processes.Methods[pid] = methods;
}
return methods;
}
private void OnSampleObjectAllocation( GCSampledObjectAllocationTraceData data )
{
if( FilterOutEvent( data ) )
return;
var typeName = GetProcessTypeName( data.ProcessID, data.TypeID );
if( data.TotalSizeForTypeSample >= 85000 )
{
var message = $"{data.ProcessID}.{data.ThreadID} - {data.TimeStampRelativeMSec,12} | Alloc {GetProcessTypeName( data.ProcessID, data.TypeID )} ({data.TotalSizeForTypeSample})";
log.info( message );
}
GetProcessAllocations( data.ProcessID )
.AddAllocation(
data.ThreadID,
(ulong)data.TotalSizeForTypeSample,
(ulong)data.ObjectCountForTypeSample,
typeName
);
}
private ProcessAllocations GetProcessAllocations( int pid )
{
if( !_processes.Allocations.TryGetValue( pid, out var allocations ) )
{
allocations = new ProcessAllocations( pid );
_processes.Allocations[pid] = allocations;
}
return allocations;
}
private void OnClrStackWalk( ClrStackWalkTraceData data )
{
if( FilterOutEvent( data ) )
return;
//var message = $"{data.ProcessID}.{data.ThreadID} - {data.TimeStampRelativeMSec,12} | {data.FrameCount} frames";
//log.info(message);
var callstack = BuildCallStack( data );
GetProcessAllocations( data.ProcessID ).AddStack( data.ThreadID, callstack );
//DumpStack(data);
}
private AddressStack BuildCallStack( ClrStackWalkTraceData data )
{
var length = data.FrameCount;
AddressStack stack = new AddressStack( length );
// frame 0 is the last frame of the stack (i.e. last called method)
for( int i = 0; i < length; i++ )
{
stack.AddFrame( data.InstructionPointer( i ) );
}
return stack;
}
private void DumpStack( ClrStackWalkTraceData data )
{
var methods = GetProcessMethods( data.ProcessID );
for( int i = 0; i < data.FrameCount; i++ )
{
var address = data.InstructionPointer( i );
log.info( methods.GetFullName( address ) );
}
log.info( $"" );
}
private void OnTypeBulkType( GCBulkTypeTraceData data )
{
if( FilterOutEvent( data ) )
return;
ProcessTypeMapping mapping = GetProcessTypesMapping( data.ProcessID );
for( int currentType = 0; currentType < data.Count; currentType++ )
{
GCBulkTypeValues value = data.Values( currentType );
mapping[value.TypeID] = value.TypeName;
}
}
private ProcessTypeMapping GetProcessTypesMapping( int pid )
{
ProcessTypeMapping mapping;
if( !_processes.Types.TryGetValue( pid, out mapping ) )
{
AssociateProcess( pid );
mapping = new ProcessTypeMapping( pid );
_processes.Types[pid] = mapping;
}
return mapping;
}
private void AssociateProcess( int pid )
{
try
{
_processes.Names[pid] = Process.GetProcessById( pid ).ProcessName;
}
catch( Exception )
{
log.info( $"? {pid}" );
// we might not have access to the process
}
}
private string GetProcessTypeName( int pid, ulong typeID )
{
if( !_processes.Types.TryGetValue( pid, out var mapping ) )
{
return typeID.ToString();
}
var name = mapping[typeID];
return string.IsNullOrEmpty( name ) ? typeID.ToString() : name;
}
private bool FilterOutEvent( TraceEvent data )
{
return ( data.ProcessID == _currentPid );
}
}
}

393
prof/MemoryPipe.cs
View File

@ -1,393 +0,0 @@
using Microsoft.Diagnostics.NETCore.Client;
using Microsoft.Diagnostics.Tracing;
using Microsoft.Diagnostics.Tracing.Etlx;
using Microsoft.Diagnostics.Tracing.Parsers;
using Microsoft.Diagnostics.Tracing.Parsers.Clr;
using Microsoft.Diagnostics.Tracing.Parsers.Kernel;
using Microsoft.Diagnostics.Tracing.Session;
using ProfilerHelpers;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Diagnostics.Tracing;
using System.Threading;
using System.Threading.Tasks;
using TraceKeywords = Microsoft.Diagnostics.Tracing.Parsers.ClrTraceEventParser.Keywords;
namespace Tracing
{
public class MemoryPipe
{
private readonly DiagnosticsClient _client;
private readonly PerProcessProfilingState _processes;
// because we are not interested in self monitoring
private readonly int _currentPid;
private int _started = 0;
Thread _thread;
public MemoryPipe()
{
_currentPid = Process.GetCurrentProcess().Id;
_client = new DiagnosticsClient( _currentPid );
_processes = new();
var threadStart = new ThreadStart( () => Start( true ) );
_thread = new Thread( threadStart );
_thread.Start();
}
//public async Task StartAsync( bool allAllocations )
public void Start( bool allAllocations )
{
if( Interlocked.CompareExchange( ref _started, 1, 0 ) == 1 )
{
throw new InvalidOperationException( "Impossible to start profiling more than once." );
}
log.info( $"Starting MemoryPipe on thread {Thread.CurrentThread.ManagedThreadId}" );
var providers = new List<EventPipeProvider>()
{
new EventPipeProvider("Microsoft-Windows-DotNETRuntime",
EventLevel.Verbose, (long)(
TraceKeywords.GC |
TraceKeywords.Contention |
TraceKeywords.Debugger |
TraceKeywords.Exception |
TraceKeywords.GCAllObjectAllocation |
TraceKeywords.GCSampledObjectAllocationHigh |
TraceKeywords.GCSampledObjectAllocationLow |
TraceKeywords.Security |
TraceKeywords.Threading |
TraceKeywords.Type |
TraceKeywords.TypeDiagnostic |
TraceKeywords.WaitHandle |
TraceKeywords.All
) )
};
//await Task.Factory.StartNew( () => {
using EventPipeSession session = _client.StartEventPipeSession( providers, false );
//log.info( $"SetupProviders" );
//SetupProviders( session, allAllocations );
var source = new EventPipeEventSource( session.EventStream );
log.info( $"SetupListeners" );
SetupListeners( source );
log.info( $"Run Process" );
source.Process();
log.info( $"Done" );
//} );
}
private void SetupProviders( TraceEventSession session, bool noSampling )
{
// Note: the kernel provider MUST be the first provider to be enabled
// If the kernel provider is not enabled, the callstacks for CLR events are still received
// but the symbols are not found (except for the application itself)
// Maybe a TraceEvent implementation details triggered when a module (image) is loaded
var success = true;
//*
log.info( $"EnableKernelProvider" );
success = log.var( session.EnableKernelProvider( KernelTraceEventParser.Keywords.None |
// KernelTraceEventParser.Keywords.ImageLoad |
// KernelTraceEventParser.Keywords.Process
0,
KernelTraceEventParser.Keywords.None
) );
log.info( $"EnableKernelProvider {success}" );
//*/
// The CLR source code indicates that the provider must be set before the monitored application starts
// Note: no real difference between High and Low
ClrTraceEventParser.Keywords eventsKeyword = noSampling
? ClrTraceEventParser.Keywords.GCSampledObjectAllocationLow | ClrTraceEventParser.Keywords.GCSampledObjectAllocationHigh
: ClrTraceEventParser.Keywords.GCSampledObjectAllocationLow
;
log.info( $"EnableProvider" );
success = log.var( session.EnableProvider(
ClrTraceEventParser.ProviderGuid,
TraceEventLevel.Verbose, // this is needed in order to receive GCSampledObjectAllocation event
(ulong)(
eventsKeyword |
// required to receive the BulkType events that allows
// mapping between the type ID received in the allocation events
ClrTraceEventParser.Keywords.GCHeapAndTypeNames |
ClrTraceEventParser.Keywords.Type |
// events related to JITed methods
ClrTraceEventParser.Keywords.Jit | // Turning on JIT events is necessary to resolve JIT compiled code
ClrTraceEventParser.Keywords.JittedMethodILToNativeMap | // This is needed if you want line number information in the stacks
ClrTraceEventParser.Keywords.Loader | // You must include loader events as well to resolve JIT compiled code.
// this is mandatory to get the callstacks in each CLR event payload.
//ClrTraceEventParser.Keywords.Stack |
0
)
) );
log.info( $"EnableProvider {success}" );
// Note: ClrRundown is not needed because only new processes will be monitored
}
private void SetupListeners( EventPipeEventSource source )
{
// register for high and low keyword
// if both are set, each allocation will trigger an event (beware performance issues...)
//source.Clr.GCSampledObjectAllocation += OnSampleObjectAllocation;
source.Clr.GCAllocationTick += OnAllocTick;
// required to receive the mapping between type ID (received in GCSampledObjectAllocation)
// and their name (received in TypeBulkType)
source.Clr.TypeBulkType += OnTypeBulkType;
// messages to get callstacks
// the correlation seems to be as "simple" as taking the last event on the same thread
source.Clr.ClrStackWalk += OnClrStackWalk;
// needed to get JITed method details
source.Clr.MethodLoadVerbose += OnMethodDetails;
source.Clr.MethodDCStartVerboseV2 += OnMethodDetails;
source.Clr.ContentionLockCreated += OnLockCreated;
source.Clr.ContentionStart += OnLockStart;
source.Clr.ContentionStop += OnLockStop;
// get notified when a module is load to map the corresponding symbols
source.Kernel.ImageLoad += OnImageLoad;
}
private void OnAllocTick( GCAllocationTickTraceData data )
{
if( FilterOutEvent( data ) )
return;
//log.info( $"*** RAW: {data}" );
//var callStack = data.CallStack();
//var caller = callStack.Caller;
//log.info( $"Call stack {callStack}" );
var thisThreadId = Thread.CurrentThread.ManagedThreadId;
var typeName = GetProcessTypeName( data.ProcessID, data.TypeID );
//if( data.TotalSizeForTypeSample >= 85000 )
{
var message = $"{data.ThreadID}/{thisThreadId} Alloc {data.ObjectSize,8} at 0x{data.Address:0000000000000000} in {data.TypeName} (or {GetProcessTypeName( data.ProcessID, data.TypeID )}) ";
log.info( message );
}
GetProcessAllocations( data.ProcessID )
.AddAllocation(
data.ThreadID,
(ulong)data.ObjectSize,
(ulong)1,
typeName
);
}
private void OnLockCreated( ContentionLockCreatedTraceData data )
{
log.info( $"{data}" );
}
private void OnLockStart( ContentionStartTraceData data )
{
log.info( $"{data}" );
}
private void OnLockStop( ContentionStopTraceData data )
{
log.info( $"{data}" );
}
private void OnImageLoad( ImageLoadTraceData data )
{
//if( FilterOutEvent( data ) )
// return;
log.info( $"{data}" );
//GetProcessMethods( data.ProcessID ).AddModule( data.FileName, data.ImageBase, data.ImageSize );
log.info( $"{data.ProcessID}.{data.ThreadID} --> {data.FileName}" );
}
private void OnMethodDetails( MethodLoadUnloadVerboseTraceData data )
{
//if( FilterOutEvent( data ) )
// return;
//log.info( $"{data}" );
// care only about jitted methods
if( !data.IsJitted )
return;
var method = GetProcessMethods( data.ProcessID )
.Add( data.MethodStartAddress, data.MethodSize, data.MethodNamespace, data.MethodName, data.MethodSignature );
//log.info( $"0x{data.MethodStartAddress.ToString( "x12" )} - {data.MethodSize,6} | {data.MethodName}" );
}
private MethodStore GetProcessMethods( int pid )
{
if( !_processes.Methods.TryGetValue( pid, out var methods ) )
{
methods = new MethodStore( pid );
_processes.Methods[pid] = methods;
}
return methods;
}
private void OnSampleObjectAllocation( GCSampledObjectAllocationTraceData data )
{
if( FilterOutEvent( data ) )
return;
//log.info( $"{data}" );
var typeName = GetProcessTypeName( data.ProcessID, data.TypeID );
//if( data.TotalSizeForTypeSample >= 85000 )
{
var message = $"{data.ProcessID}.{data.ThreadID} - {data.TimeStampRelativeMSec,12} | Alloc {GetProcessTypeName( data.ProcessID, data.TypeID )} ({data.TotalSizeForTypeSample})";
log.info( message );
}
GetProcessAllocations( data.ProcessID )
.AddAllocation(
data.ThreadID,
(ulong)data.TotalSizeForTypeSample,
(ulong)data.ObjectCountForTypeSample,
typeName
);
}
private ProcessAllocations GetProcessAllocations( int pid )
{
if( !_processes.Allocations.TryGetValue( pid, out var allocations ) )
{
allocations = new ProcessAllocations( pid );
_processes.Allocations[pid] = allocations;
}
return allocations;
}
private void OnClrStackWalk( ClrStackWalkTraceData data )
{
var message = $"{data.ProcessID}.{data.ThreadID} - {data.TimeStampRelativeMSec,12} | {data.FrameCount} frames";
log.info( message );
var callstack = BuildCallStack( data );
GetProcessAllocations( data.ProcessID ).AddStack( data.ThreadID, callstack );
//DumpStack(data);
}
private AddressStack BuildCallStack( ClrStackWalkTraceData data )
{
//log.info( $"{data}" );
var length = data.FrameCount;
AddressStack stack = new AddressStack( length );
// frame 0 is the last frame of the stack (i.e. last called method)
for( int i = 0; i < length; i++ )
{
stack.AddFrame( data.InstructionPointer( i ) );
}
return stack;
}
private void DumpStack( ClrStackWalkTraceData data )
{
if( FilterOutEvent( data ) )
return;
log.info( $"{data}" );
var methods = GetProcessMethods( data.ProcessID );
for( int i = 0; i < data.FrameCount; i++ )
{
var address = data.InstructionPointer( i );
log.info( methods.GetFullName( address ) );
}
log.info( $"" );
}
private void OnTypeBulkType( GCBulkTypeTraceData data )
{
ProcessTypeMapping mapping = GetProcessTypesMapping( data.ProcessID );
for( int currentType = 0; currentType < data.Count; currentType++ )
{
GCBulkTypeValues value = data.Values( currentType );
mapping[value.TypeID] = value.TypeName;
//log.info( $"{value}" );
}
}
private ProcessTypeMapping GetProcessTypesMapping( int pid )
{
ProcessTypeMapping mapping;
if( !_processes.Types.TryGetValue( pid, out mapping ) )
{
AssociateProcess( pid );
mapping = new ProcessTypeMapping( pid );
_processes.Types[pid] = mapping;
}
return mapping;
}
private void AssociateProcess( int pid )
{
try
{
_processes.Names[pid] = Process.GetProcessById( pid ).ProcessName;
}
catch( Exception )
{
log.info( $"? {pid}" );
// we might not have access to the process
}
}
private string GetProcessTypeName( int pid, ulong typeID )
{
if( !_processes.Types.TryGetValue( pid, out var mapping ) )
{
return typeID.ToString();
}
var name = mapping[typeID];
return string.IsNullOrEmpty( name ) ? typeID.ToString() : name;
}
private bool FilterOutEvent( TraceEvent data )
{
return data.ThreadID == Thread.CurrentThread.ManagedThreadId;
//return false; // ( data.ProcessID == _currentPid );
}
}
}

68
prof/MethodInfo.cs
View File

@ -1,68 +0,0 @@
using System;
namespace ProfilerHelpers
{
public class MethodInfo
{
private readonly ulong _startAddress;
private readonly int _size;
private readonly string _fullName;
internal MethodInfo( ulong startAddress, int size, string namespaceAndTypeName, string name, string signature )
{
_startAddress = startAddress;
_size = size;
_fullName = ComputeFullName( startAddress, namespaceAndTypeName, name, signature );
}
private string ComputeFullName( ulong startAddress, string namespaceAndTypeName, string name, string signature )
{
var fullName = signature;
// constructor case: name = .ctor | namespaceAndTypeName = A.B.typeName | signature = ... (parameters)
// --> A.B.typeName(parameters)
if( name == ".ctor" )
{
return $"{namespaceAndTypeName}{ExtractParameters( signature )}";
}
// general case: name = Foo | namespaceAndTypeName = A.B.typeName | signature = ... (parameters)
// --> A.B.Foo(parameters)
fullName = $"{namespaceAndTypeName}.{name}{ExtractParameters( signature )}";
return fullName;
}
private string ExtractTypeName( string namespaceAndTypeName )
{
var pos = namespaceAndTypeName.LastIndexOf( ".", StringComparison.Ordinal );
if( pos == -1 )
{
return namespaceAndTypeName;
}
// skip the .
pos++;
return namespaceAndTypeName.Substring( pos );
}
private string ExtractParameters( string signature )
{
var pos = signature.IndexOf( " (" );
if( pos == -1 )
{
return "(???)";
}
// skip double space
pos += 2;
var parameters = signature.Substring( pos );
return parameters;
}
public ulong StartAddress => _startAddress;
public int Size => _size;
public string FullName => _fullName;
}
}

209
prof/MethodStore.cs
View File

@ -1,209 +0,0 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Text;
namespace ProfilerHelpers
{
public class MethodStore : IDisposable
{
// JITed methods information (start address + size + signature)
private readonly List<MethodInfo> _methods;
// addresses from callstacks already matching (address -> full name)
private readonly Dictionary<ulong, string> _cache;
// for native methods, rely on dbghelp API
// so the process handle is required
private IntPtr _hProcess;
private Process _process;
private readonly int _pid;
public MethodStore( int pid, bool loadModules = false )
{
// it may be possible to open the process
// in that case, _hProcess = IntPtr.Zero
_pid = pid;
_methods = new List<MethodInfo>( 1024 );
_cache = new Dictionary<ulong, string>();
_hProcess = BindToProcess( pid, loadModules );
}
private IntPtr BindToProcess( int pid, bool loadModules )
{
try
{
_process = Process.GetProcessById( pid );
if( !SymInitialize( _process.Handle, loadModules ) )
return IntPtr.Zero;
return _process.Handle;
}
catch( Exception x )
{
Console.WriteLine( $"Error while binding pid #{pid} to DbgHelp:" );
Console.WriteLine( x.Message );
return IntPtr.Zero;
}
}
private bool SymInitialize( IntPtr hProcess, bool loadModules = false )
{
// read https://docs.microsoft.com/en-us/windows/win32/api/dbghelp/nf-dbghelp-symsetoptions for more details
// maybe SYMOPT_NO_PROMPTS and SYMOPT_FAIL_CRITICAL_ERRORS could be used
NativeDbgHelp.SymSetOptions(
NativeDbgHelp.SYMOPT_DEFERRED_LOADS | // performance optimization
NativeDbgHelp.SYMOPT_UNDNAME // C++ names are not mangled
);
// https://docs.microsoft.com/en-us/windows/win32/api/dbghelp/nf-dbghelp-syminitialize
// search path for symbols:
// - The current working directory of the application
// - The _NT_SYMBOL_PATH environment variable
// - The _NT_ALTERNATE_SYMBOL_PATH environment variable
//
// passing false as last parameter means that we will need to call SymLoadModule64
// each time a module is loaded in the process
return NativeDbgHelp.SymInitialize( hProcess, null, loadModules );
}
public MethodInfo Add( ulong address, int size, string namespaceAndTypeName, string name, string signature )
{
var method = new MethodInfo( address, size, namespaceAndTypeName, name, signature );
_methods.Add( method );
return method;
}
public string GetFullName( ulong address )
{
if( _cache.TryGetValue( address, out var fullName ) )
return fullName;
// look for managed methods
for( int i = 0; i < _methods.Count; i++ )
{
var method = _methods[i];
if( ( address >= method.StartAddress ) && ( address < method.StartAddress + (ulong)method.Size ) )
{
fullName = method.FullName;
_cache[address] = fullName;
return fullName;
}
}
// look for native methods
fullName = GetNativeMethodName( address );
_cache[address] = fullName;
return fullName;
}
private string GetNativeMethodName( ulong address )
{
var symbol = new NativeDbgHelp.SYMBOL_INFO();
symbol.MaxNameLen = 1024;
symbol.SizeOfStruct = (uint)Marshal.SizeOf( symbol ) - 1024; // char buffer is not counted
// the ANSI version of SymFromAddr is called so each character is 1 byte long
if( NativeDbgHelp.SymFromAddr( _hProcess, address, out var displacement, ref symbol ) )
{
var buffer = new StringBuilder( symbol.Name.Length );
// remove weird "$##" at the end of some symbols
var pos = symbol.Name.LastIndexOf( "$##" );
if( pos == -1 )
buffer.Append( symbol.Name );
else
buffer.Append( symbol.Name, 0, pos );
// add offset if any
if( displacement != 0 )
buffer.Append( $"+0x{displacement}" );
return buffer.ToString();
}
// default value is just the address in HEX
#if DEBUG
return ( $"0x{address:x} (SymFromAddr failed with 0x{Marshal.GetLastWin32Error():x})" );
#else
return $"0x{address:x}";
#endif
}
const int ERROR_SUCCESS = 0;
public void AddModule( string filename, ulong baseOfDll, int sizeOfDll )
{
var baseAddress = NativeDbgHelp.SymLoadModule64( _hProcess, IntPtr.Zero, filename, null, baseOfDll, (uint)sizeOfDll );
if( baseAddress == 0 )
{
// should work if the same module is added more than once
if( Marshal.GetLastWin32Error() == ERROR_SUCCESS )
return;
Console.WriteLine( $"SymLoadModule64 failed for {filename}" );
}
}
public void Dispose()
{
if( _hProcess == IntPtr.Zero )
return;
_hProcess = IntPtr.Zero;
_process.Dispose();
}
}
internal static class NativeDbgHelp
{
// from C:\Program Files (x86)\Windows Kits\10\Debuggers\inc\dbghelp.h
public const uint SYMOPT_UNDNAME = 0x00000002;
public const uint SYMOPT_DEFERRED_LOADS = 0x00000004;
[StructLayout( LayoutKind.Sequential )]
public struct SYMBOL_INFO
{
public uint SizeOfStruct;
public uint TypeIndex; // Type Index of symbol
private ulong Reserved1;
private ulong Reserved2;
public uint Index;
public uint Size;
public ulong ModBase; // Base Address of module containing this symbol
public uint Flags;
public ulong Value; // Value of symbol, ValuePresent should be 1
public ulong Address; // Address of symbol including base address of module
public uint Register; // register holding value or pointer to value
public uint Scope; // scope of the symbol
public uint Tag; // pdb classification
public uint NameLen; // Actual length of name
public uint MaxNameLen;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst = 1024 )]
public string Name;
}
[DllImport( "dbghelp.dll", SetLastError = true )]
public static extern bool SymInitialize( IntPtr hProcess, string userSearchPath, bool invadeProcess );
[DllImport( "dbghelp.dll", SetLastError = true )]
public static extern uint SymSetOptions( uint symOptions );
[DllImport( "dbghelp.dll", SetLastError = true, CharSet = CharSet.Ansi )]
public static extern ulong SymLoadModule64( IntPtr hProcess, IntPtr hFile, string imageName, string moduleName, ulong baseOfDll, uint sizeOfDll );
// use ANSI version to ensure the right size of the structure
// read https://docs.microsoft.com/en-us/windows/win32/api/dbghelp/ns-dbghelp-symbol_info
[DllImport( "dbghelp.dll", SetLastError = true, CharSet = CharSet.Ansi )]
public static extern bool SymFromAddr( IntPtr hProcess, ulong address, out ulong displacement, ref SYMBOL_INFO symbol );
[DllImport( "dbghelp.dll", SetLastError = true )]
public static extern bool SymCleanup( IntPtr hProcess );
}
}

33
prof/PerProcessProfilingState.cs
View File

@ -1,33 +0,0 @@
using ProfilerHelpers;
using System;
using System.Collections.Generic;
namespace Tracing
{
public class PerProcessProfilingState : IDisposable
{
private bool _disposed;
private readonly Dictionary<int, string> _processNames = new Dictionary<int, string>();
private readonly Dictionary<int, ProcessTypeMapping> _perProcessTypes = new Dictionary<int, ProcessTypeMapping>();
private readonly Dictionary<int, ProcessAllocations> _perProcessAllocations = new Dictionary<int, ProcessAllocations>();
private readonly Dictionary<int, MethodStore> _methods = new Dictionary<int, MethodStore>();
public Dictionary<int, string> Names => _processNames;
public Dictionary<int, ProcessTypeMapping> Types => _perProcessTypes;
public Dictionary<int, ProcessAllocations> Allocations => _perProcessAllocations;
public Dictionary<int, MethodStore> Methods => _methods;
public void Dispose()
{
if( _disposed )
return;
_disposed = true;
foreach( var methodStore in _methods.Values )
{
methodStore.Dispose();
}
}
}
}

131
prof/ProcessAllocations.cs
View File

@ -1,131 +0,0 @@
using System;
using System.Collections.Generic;
namespace Tracing
{
public class ProcessAllocations
{
private readonly int _pid;
private readonly Dictionary<string, AllocationInfo> _allocations;
private readonly Dictionary<int, AllocationInfo> _perThreadLastAllocation;
public ProcessAllocations( int pid )
{
_pid = pid;
_allocations = new Dictionary<string, AllocationInfo>();
_perThreadLastAllocation = new Dictionary<int, AllocationInfo>();
}
public int Pid => _pid;
public AllocationInfo GetAllocations( string typeName )
{
return ( _allocations.TryGetValue( typeName, out var info ) ) ? info : null;
}
public IEnumerable<AllocationInfo> GetAllAllocations()
{
return _allocations.Values;
}
public AllocationInfo AddAllocation( int threadID, ulong size, ulong count, string typeName )
{
if( !_allocations.TryGetValue( typeName, out var info ) )
{
info = new AllocationInfo( typeName );
_allocations[typeName] = info;
}
info.AddAllocation( size, count );
// the last allocation is still here without the corresponding stack
if( _perThreadLastAllocation.TryGetValue( threadID, out var lastAlloc ) )
{
Console.WriteLine( "no stack for the last allocation" );
}
// keep track of the allocation for the given thread
// --> will be used when the corresponding call stack event will be received
_perThreadLastAllocation[threadID] = info;
return info;
}
public void AddStack( int threadID, AddressStack stack )
{
if( _perThreadLastAllocation.TryGetValue( threadID, out var lastAlloc ) )
{
lastAlloc.AddStack( stack );
_perThreadLastAllocation.Remove( threadID );
return;
}
//Console.WriteLine("no last allocation for the stack event");
}
}
public class AllocationInfo
{
private readonly string _typeName;
private ulong _size;
private ulong _count;
private List<StackInfo> _stacks;
internal AllocationInfo( string typeName )
{
_typeName = typeName;
_stacks = new List<StackInfo>();
}
public string TypeName => _typeName;
public ulong Count => _count;
public ulong Size => _size;
public IReadOnlyList<StackInfo> Stacks => _stacks;
internal void AddAllocation( ulong size, ulong count )
{
_count += count;
_size += size;
}
internal void AddStack( AddressStack stack )
{
var info = GetInfo( stack );
if( info == null )
{
info = new StackInfo( stack );
_stacks.Add( info );
}
info.Count++;
}
private StackInfo GetInfo( AddressStack stack )
{
for( int i = 0; i < _stacks.Count; i++ )
{
var info = _stacks[i];
if( stack.Equals( info.Stack ) )
return info;
}
return null;
}
}
public class StackInfo
{
private readonly AddressStack _stack;
public ulong Count;
internal StackInfo( AddressStack stack )
{
Count = 0;
_stack = stack;
}
public AddressStack Stack => _stack;
}
}

35
prof/ProcessTypeMapping.cs
View File

@ -1,35 +0,0 @@
using System.Collections.Generic;
namespace Tracing
{
// Contains the mapping between type ID received by SampleObjectAllocation(Low/High) events
// and their name received by TypeBulkType events
public class ProcessTypeMapping
{
private readonly Dictionary<ulong, string> _typesIdToName;
public ProcessTypeMapping( int processId )
{
ProcessId = processId;
_typesIdToName = new Dictionary<ulong, string>();
}
public int ProcessId { get; set; }
public string this[ulong id]
{
get
{
if( !_typesIdToName.ContainsKey( id ) )
return null;
return _typesIdToName[id];
}
set
{
_typesIdToName[id] = value;
}
}
}
}

162
prof/Program.cs
View File

@ -1,162 +0,0 @@
using Microsoft.Diagnostics.Tracing.Session;
using ProfilerHelpers;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
namespace Tracing
{
static class Program
{
static public int CreateTracingSession( bool noSampling, bool sortBySize, int topTypesLimit )
{
ShowHeader();
try
{
TraceEventSession session = new TraceEventSession(
"Tracing",
TraceEventSessionOptions.Create
);
log.info( $"Create PerProcessProfilingState" );
using( var processes = new PerProcessProfilingState() )
{
log.info( $"Create Memory profiler for session" );
var profiler = new Memory( session, processes );
log.info( $"Start task" );
var task = profiler.StartAsync( noSampling );
/*
log.info( $"await the Continue" );
await task.ContinueWith( ( t ) => {
log.info( $"Task is done, Dispose" );
session.Dispose();
} );
*/
//log.info("Press ENTER to stop memory profiling");
//Console.ReadLine();
/*
try
{
await task;
ShowResults( processes, sortBySize, topTypesLimit );
return 0;
}
catch( Exception x )
{
log.info( x.Message );
ShowHelp();
}
*/
}
return -1;
}
catch( Exception x )
{
log.info( x.Message );
ShowHelp();
}
return -2;
}
private static void ShowResults( PerProcessProfilingState processes, bool sortBySize, int topTypesLimit )
{
foreach( var pid in processes.Allocations.Keys )
{
// skip processes without symbol resolution
if( !processes.Methods.ContainsKey( pid ) )
continue;
// skip processes without allocations
if( !processes.Allocations[pid].GetAllAllocations().Any() )
continue;
ShowResults( GetProcessName( pid, processes.Names ), processes.Methods[pid], processes.Allocations[pid], sortBySize, topTypesLimit );
}
}
private static string GetProcessName( int pid, Dictionary<int, string> names )
{
if( names.TryGetValue( pid, out var name ) )
return name;
return pid.ToString();
}
private static void ShowResults( string name, MethodStore methods, ProcessAllocations allocations, bool sortBySize, int topTypesLimit )
{
log.info( $"Memory allocations for {name}" );
log.info( $"" );
log.info( "---------------------------------------------------------" );
log.info( " Count Size Type" );
log.info( "---------------------------------------------------------" );
IEnumerable<AllocationInfo> types = ( sortBySize )
? allocations.GetAllAllocations().OrderByDescending( a => a.Size )
: allocations.GetAllAllocations().OrderByDescending( a => a.Count )
;
if( topTypesLimit != -1 )
types = types.Take( topTypesLimit );
foreach( var allocation in types )
{
log.info( $"{allocation.Count,9} {allocation.Size,11} {allocation.TypeName}" );
log.info( $"" );
DumpStacks( allocation, methods );
log.info( $"" );
}
log.info( $"" );
log.info( $"" );
}
private static void DumpStacks( AllocationInfo allocation, MethodStore methods )
{
var stacks = allocation.Stacks.OrderByDescending( s => s.Count ).Take( 10 );
foreach( var stack in stacks )
{
log.info( $"{stack.Count,6} allocations" );
log.info( "----------------------------------" );
DumpStack( stack.Stack, methods );
log.info( $"" );
}
}
private static void DumpStack( AddressStack stack, MethodStore methods )
{
var callstack = stack.Stack;
for( int i = 0; i < Math.Min( 10, callstack.Count ); i++ )
{
log.info( $" {methods.GetFullName( callstack[i] )}" );
}
}
private static void ShowHeader()
{
log.info( "Tracing v1.0.0 - Sampled memory profiler for .NET applications" );
log.info( "by Christophe Nasarre" );
log.info( $"" );
}
private static void ShowHelp()
{
log.info( $"" );
log.info( "Tracing shows sampled allocations of a given .NET application." );
log.info( "Usage: Tracing [-a (all allocations)] [-c (sort by count instead of default by size)] [-t <type count (instead of 3 types by default)>]" );
log.info( " Ex: Tracing -t -1 (all types sampled allocations sorted by size)" );
log.info( " Ex: Tracing -c -t 10 (allocations for top 10 types sorted by count)" );
log.info( $"" );
}
}
}

106
reflect/refl.cs
View File

@ -4,6 +4,7 @@ using System.Collections.Generic;
using System.Collections.Immutable;
using System.Linq;
using System.Reflection;
using System.Runtime.Serialization;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
@ -18,6 +19,28 @@ using System.Threading.Tasks;
static public class refl
{
static public T Create<T>()
{
var t = typeof( T );
return (T)CreateObject( t );
}
static public object CreateObject( Type type )
{
object newObj = null;
if( type.GetConstructor( Type.EmptyTypes ) != null )
{
newObj = Activator.CreateInstance( type );
}
else
{
newObj = FormatterServices.GetUninitializedObject( type );
}
return newObj;
}
public class PredEnumerator
{
public static PredEnumerator<T> Create<T>( IEnumerator<T> en, Predicate<T> pred )
@ -142,6 +165,83 @@ static public class refl
}
public static void GetAllMembers( Type t, List<MemberInfo> list )
{
GetAllMembersRecursive( t, true, list );
}
public static void GetAllMembersRecursive( Type t, bool recurse, List<MemberInfo> list )
{
var MemberArr = t.GetMembers(
BindingFlags.DeclaredOnly |
BindingFlags.NonPublic |
BindingFlags.Public |
BindingFlags.Instance );
var en = PredEnumerator.Create<MemberInfo>( MemberArr.AsEnumerable<MemberInfo>(), fa => fa.GetCustomAttribute( typeof( NonSerializedAttribute ) ) == null );
list.AddRange( new PredEnumerable<MemberInfo>( en ) );
if( recurse && t.BaseType != null && t.BaseType != typeof( object ) )
{
GetAllMembers( t.BaseType, list );
}
}
public static void GetAllMembersUntil( Type t, Type tooFar, List<MemberInfo> list )
{
var MemberArr = t.GetMembers(
BindingFlags.DeclaredOnly |
BindingFlags.NonPublic |
BindingFlags.Public |
BindingFlags.Instance );
var en = PredEnumerator.Create<MemberInfo>( MemberArr.AsEnumerable<MemberInfo>(), fa => fa.GetCustomAttribute( typeof( NonSerializedAttribute ) ) == null );
list.AddRange( new PredEnumerable<MemberInfo>( en ) );
if( t.BaseType != null && t.BaseType != tooFar )
{
GetAllMembers( t.BaseType, list );
}
}
public static ImmutableList<MemberInfo> GetAllMembers<T>()
{
return GetAllMembers( typeof( T ) );
}
public static ImmutableList<MemberInfo> GetAllMembers( Type t )
{
{
if( s_MemberCache.TryGetValue( t, out var first ) )
return first;
}
//LogGC.RegisterObjectId( t );
lock( t )
{
if( s_MemberCache.TryGetValue( t, out var second ) )
return second;
var list = new List<MemberInfo>();
GetAllMembers( t, list );
var immList = list.ToImmutableList();
Interlocked.Exchange( ref s_MemberCache, s_MemberCache.Add( t, immList ) );
return immList;
}
}
public static void GetAllFields( Type t, List<FieldInfo> list )
{
GetAllFieldsRecursive( t, true, list );
@ -164,6 +264,7 @@ static public class refl
GetAllFields( t.BaseType, list );
}
}
public static void GetAllFieldsUntil( Type t, Type tooFar, List<FieldInfo> list )
{
var fieldArr = t.GetFields(
@ -187,8 +288,6 @@ static public class refl
return GetAllFields( typeof( T ) );
}
public static ImmutableList<FieldInfo> GetAllFields( Type t )
{
{
@ -196,7 +295,7 @@ static public class refl
return first;
}
LogGC.RegisterObjectId( t );
//LogGC.RegisterObjectId( t );
lock( t )
{
@ -253,6 +352,7 @@ static public class refl
}
static ImmutableDictionary<Type, ImmutableList<MemberInfo>> s_MemberCache = ImmutableDictionary<Type, ImmutableList<MemberInfo>>.Empty;
static ImmutableDictionary<Type, ImmutableList<FieldInfo>> s_fieldCache = ImmutableDictionary<Type, ImmutableList<FieldInfo>>.Empty;
static ImmutableDictionary<Type, ImmutableList<PropertyInfo>> s_propCache = ImmutableDictionary<Type, ImmutableList<PropertyInfo>>.Empty;

97
res/Resource.cs
View File

@ -6,13 +6,18 @@ using System.Reflection;
using System.Collections.Immutable;
using System.Runtime.CompilerServices;
using System.Threading;
using Microsoft.CodeAnalysis; // Note: This was in the original, but seems unused. Keep for now.
using System.Linq;
using System.Text.Json.Serialization;
#nullable enable
namespace res;
public interface Resource
{
public string Filename { get; set; }
}
// A delegate representing a function that can load a resource of type T.
public delegate T Load<out T>( string filename );
@ -34,13 +39,13 @@ public abstract class Ref
protected Ref(
string filename,
string reason = "",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0 )
{
Filename = filename;
Reason = reason;
DbgName = dbgName;
DbgName = dbgMethod;
DbgPath = dbgPath;
DbgLine = dbgLine;
@ -54,7 +59,7 @@ public abstract class Ref
/// <returns>The loaded resource object.</returns>
public abstract object Lookup(
string reason = "",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0 );
@ -67,6 +72,41 @@ public abstract class Ref
/// Internal method to trigger the initial load (used by deferred loading, if implemented).
/// </summary>
internal virtual void InternalLoad() { }
static public res.Ref<T> Create<T>( T res, [CallerFilePath] string dbgPath = "", [CallerLineNumber] int dbgLine = -1, [CallerMemberName] string dbgMethod = "" ) where T : class, Resource, new()
{
if( res.Filename == null )
{
res.Filename = $"{{{typeof( T ).Name}}}.{Ref<T>.s_codeRes}";
log.debug( $"Code Res for {typeof( T ).Name} named {res.Filename}" );
}
return Ref<T>.CreateAsset( res, res.Filename );
}
static public res.Ref<T> Create<T>( string filename, T res, [CallerFilePath] string dbgPath = "", [CallerLineNumber] int dbgLine = -1, [CallerMemberName] string dbgMethod = "" ) where T : class, new()
{
if( string.IsNullOrWhiteSpace( filename ) )
{
filename = $"{{{typeof( T ).Name}}}.{Ref<T>.s_codeRes}";
log.debug( $"Code Res for {typeof( T ).Name} named {filename}" );
}
return Ref<T>.CreateAsset( res, filename );
}
static public res.Ref<T> Create<T>( string filename, [CallerFilePath] string dbgPath = "", [CallerLineNumber] int dbgLine = -1, [CallerMemberName] string dbgMethod = "" ) where T : class, new()
{
if( string.IsNullOrWhiteSpace( filename ) )
{
filename = $"{{{typeof( T ).Name}}}.{Ref<T>.s_codeRes}";
log.debug( $"Code Res for {typeof( T ).Name} named {filename}" );
}
var res = refl.Create<T>();
return Ref<T>.CreateAsset( res, filename );
}
}
/// <summary>
@ -77,6 +117,9 @@ public abstract class Ref
[DebuggerDisplay( "Path = {Filename} / Res = {m_res}" )]
public class Ref<T> : Ref where T : class, new()
{
static internal int s_codeRes = 1024;
[JsonIgnore]
[NonSerialized]
private T? m_res;
@ -89,17 +132,18 @@ public class Ref<T> : Ref where T : class, new()
/// Gets the resource, loading it if necessary.
/// </summary>
//[Deprecated("Use Res property instead.")]
[JsonIgnore]
public T res => m_res ?? Lookup();
public Ref(
string filename = "",
string reason = "",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0 )
: base(
!string.IsNullOrWhiteSpace( filename ) ? filename : $"{{{dbgName}_{Path.GetFileNameWithoutExtension( dbgPath )}}}",
reason, dbgName, dbgPath, dbgLine )
!string.IsNullOrWhiteSpace( filename ) ? filename : $"{{{dbgMethod}_{Path.GetFileNameWithoutExtension( dbgPath )}}}",
reason, dbgMethod, dbgPath, dbgLine )
{
if( VerboseLogging )
log.info( $"Ref<T> Created: {GetType().Name}<{typeof( T ).Name}> {Filename}" );
@ -111,7 +155,7 @@ public class Ref<T> : Ref where T : class, new()
/// </summary>
public override T Lookup(
string reason = "",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0 )
{
@ -120,7 +164,7 @@ public class Ref<T> : Ref where T : class, new()
return m_res;
// Load using the Mgr.
m_res = Mgr.Load<T>( Filename, $"Ref lookup (orig: {Reason}) bcs {reason}", dbgName, dbgPath, dbgLine );
m_res = Mgr.Load<T>( Filename, $"Ref lookup (orig: {Reason}) bcs {reason}", dbgMethod, dbgPath, dbgLine );
if( VerboseLogging )
log.info( $"Ref.Lookup: {GetType().Name}<{typeof( T ).Name}> {Filename}" );
return m_res;
@ -132,11 +176,11 @@ public class Ref<T> : Ref where T : class, new()
/// </summary>
public override object Lookup(
string reason = "",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0)
{
return Lookup(reason, dbgName, dbgPath, dbgLine);
return Lookup(reason, dbgMethod, dbgPath, dbgLine);
}
*/
@ -160,7 +204,7 @@ public class Ref<T> : Ref where T : class, new()
public static Ref<T> CreateAsset(
T value, string path,
string reason = "",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0 )
{
@ -187,7 +231,7 @@ public class Ref<T> : Ref where T : class, new()
var createReason = $"CreateAsset: {value.GetType().Name} - {immMeta?.Reason ?? "N/A"}";
var newRef = new Ref<T>( path, $"{createReason} bcs {reason}", dbgName, dbgPath, dbgLine );
var newRef = new Ref<T>( path, $"{createReason} bcs {reason}", dbgMethod, dbgPath, dbgLine );
// We should make the newRef hold the 'value' immediately,
// or ensure loading it back gives the same 'value'.
@ -210,6 +254,9 @@ internal record ResourceHolder<T>( WeakReference<T> WeakRef, string Name, DateTi
/// </summary>
public static class Mgr
{
// Internal holder for type-specific loaders.
private abstract class LoadHolder
{
@ -217,7 +264,7 @@ public static class Mgr
// Its not yet working, and maybe shouldnt exist
public abstract object Load( string filename,
string reason = "",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0
);
@ -228,7 +275,7 @@ public static class Mgr
private readonly Load<T> _fnLoad;
public LoadHolder( Load<T> fnLoad ) { _fnLoad = fnLoad; }
public override object Load( string filename, string reason = "",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0
) => _fnLoad( filename )!;
@ -336,27 +383,26 @@ public static class Mgr
}
}
/// <summary>
/// Creates a Ref<T> for a given filename.
/// </summary>
public static Ref<T> Lookup<T>(
string filename,
string reason = "",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0 ) where T : class, new()
{
return new Ref<T>( filename, reason, dbgName, dbgPath, dbgLine );
return new Ref<T>( filename, reason, dbgMethod, dbgPath, dbgLine );
}
/// <summary>
/// Loads a resource, handling caching and thread-safe loading.
/// </summary>
public static T Load<T>(
internal static T Load<T>(
string filename,
string reason = "",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0 ) where T : class, new()
{
@ -377,8 +423,8 @@ public static class Mgr
}
// 4. Perform the actual load
log.warn( $"Loading {typeof( T ).Name}: {filename} ({reason} at {dbgName}:{dbgLine})" );
var newValue = ActualLoad<T>( filename, reason, dbgName, dbgPath, dbgLine );
log.warn( $"Loading {typeof( T ).Name}: {filename} ({reason} at {dbgMethod}:{dbgLine})" );
var newValue = ActualLoad<T>( filename, reason, dbgMethod, dbgPath, dbgLine );
// 5. Cache the new value
CacheResource( filename, newValue, reason );
@ -398,7 +444,6 @@ public static class Mgr
log.info( $"Cached {typeof( T ).Name}: {filename} ({reason})" );
}
/// <summary>
/// Tries to retrieve a resource from the cache.
/// </summary>
@ -423,14 +468,14 @@ public static class Mgr
private static T ActualLoad<T>(
string filename,
string reason = "",
[CallerMemberName] string dbgName = "",
[CallerMemberName] string dbgMethod = "",
[CallerFilePath] string dbgPath = "",
[CallerLineNumber] int dbgLine = 0
) where T : class, new()
{
if( s_loaders.TryGetValue( typeof( T ), out var loaderHolder ) )
{
var loadedObject = loaderHolder.Load( filename, reason, dbgName, dbgPath, dbgLine );
var loadedObject = loaderHolder.Load( filename, reason, dbgMethod, dbgPath, dbgLine );
if( loadedObject is T value )
{
var meta = ( value as io.Obj )?.Meta;
@ -438,7 +483,7 @@ public static class Mgr
// If it's an immutable object, record its loading.
if( value is io.Obj imm )
{
return (T)imm.Record( $"Loading bcs {reason}", dbgName, dbgPath, dbgLine );
return (T)imm.Record( $"Loading bcs {reason}", dbgMethod, dbgPath, dbgLine );
}
return value;
}

62
scr/Script.cs
View File

@ -3,7 +3,7 @@
#nullable enable
#if SCRIPTS
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp;
using Microsoft.CodeAnalysis.Emit;
@ -20,43 +20,64 @@ using System.Threading;
public class MemorySourceText : SourceText
{
public override char this[int position] => ' ';
private readonly string _content;
public MemorySourceText(string content = "")
{
_content = content ?? string.Empty;
}
public override char this[int position] => _content[position];
public override Encoding? Encoding => Encoding.UTF8;
public override int Length => 0;
public override int Length => _content.Length;
public override void CopyTo( int sourceIndex, char[] destination, int destinationIndex, int count )
public override void CopyTo(int sourceIndex, char[] destination, int destinationIndex, int count)
{
if (destination == null)
throw new ArgumentNullException(nameof(destination));
if (sourceIndex < 0 || sourceIndex >= _content.Length)
throw new ArgumentOutOfRangeException(nameof(sourceIndex));
if (destinationIndex < 0 || destinationIndex >= destination.Length)
throw new ArgumentOutOfRangeException(nameof(destinationIndex));
if (count < 0)
throw new ArgumentOutOfRangeException(nameof(count));
if (sourceIndex + count > _content.Length || destinationIndex + count > destination.Length)
throw new ArgumentException("Source or destination range is out of bounds.");
_content.CopyTo(sourceIndex, destination, destinationIndex, count);
}
}
public class MemoryRefResolver : SourceReferenceResolver
{
public override bool Equals( object? other )
public override bool Equals(object? other)
{
return false;
return other is MemoryRefResolver;
}
public override int GetHashCode()
{
return 0;
return typeof(MemoryRefResolver).GetHashCode();
}
private MemoryStream _fakeMS = new MemoryStream();
private readonly MemoryStream _fakeMS = new MemoryStream();
public override string? NormalizePath( string? path, string? baseFilePath ) => path;
public override string? NormalizePath(string? path, string? baseFilePath) => path;
public override Stream OpenRead( string resolvedPath ) => _fakeMS;
public override Stream OpenRead(string resolvedPath) => _fakeMS;
private MemorySourceText _fakeMST = new();
private readonly MemorySourceText _fakeMST = new();
public override SourceText ReadText( string resolvedPath ) => _fakeMST;
public override string? ResolveReference( string? path, string? baseFilePath ) => path;
public override SourceText ReadText(string resolvedPath) => _fakeMST;
public override string? ResolveReference(string? path, string? baseFilePath) => path;
}
@ -204,7 +225,7 @@ public static class scr
CompileFile( filename, ( ass ) => { s_fnAss( ass ); }, ( diags ) => { } );
}
public static void CompileFile( string filename, Action<Assembly> onSuccess, Action<ImmutableArray<Diagnostic>> onFailure, Platform platform = Platform.X86 )
public static void CompileFile( string filename, Action<Assembly> onSuccess, Action<ImmutableArray<Diagnostic>> onFailure/* PORT , Platform platform = Platform.X86*/ )
{
var fullpath = Path.GetFullPath( filename );
@ -215,7 +236,7 @@ public static class scr
Compile( sourceText, fullpath, onSuccess, onFailure );
}
public static void Compile( string str, string uniquePath, Action<Assembly> onSuccess, Action<ImmutableArray<Diagnostic>> onFailure, Platform platform = Platform.X86 )
public static void Compile( string str, string uniquePath, Action<Assembly> onSuccess, Action<ImmutableArray<Diagnostic>> onFailure /*, Platform platform = Platform.X86*/ )
{
var sourceText = SourceText.From( str );
@ -223,7 +244,7 @@ public static class scr
}
public static void Compile( SourceText sourceText, string uniquePath, Action<Assembly> onSuccess, Action<ImmutableArray<Diagnostic>> onFailure, Platform platform = Platform.X86 )
public static void Compile( SourceText sourceText, string uniquePath, Action<Assembly> onSuccess, Action<ImmutableArray<Diagnostic>> onFailure /*, Platform platform = Platform.X86*/ )
{
string assemblyName = Path.GetRandomFileName();
@ -237,7 +258,7 @@ public static class scr
using MemoryStream ms = new();
using MemoryStream pdb = new();
var result = CompileAndEmit( assemblyName, new[] { syntaxTree }, ms, pdb, platform );
var result = CompileAndEmit( assemblyName, new[] { syntaxTree }, ms, pdb );
if( !result.Success )
{
@ -269,7 +290,7 @@ public static class scr
}
}
private static EmitResult CompileAndEmit( string assemblyName, SyntaxTree[] syntaxTrees, MemoryStream ms, MemoryStream pdb, Platform platform )
private static EmitResult CompileAndEmit( string assemblyName, SyntaxTree[] syntaxTrees, MemoryStream ms, MemoryStream pdb /*, Platform platform*/ )
{
MemoryRefResolver memRef = new();
@ -281,7 +302,7 @@ public static class scr
options: new CSharpCompilationOptions( OutputKind.DynamicallyLinkedLibrary,
sourceReferenceResolver: memRef,
optimizationLevel: OptimizationLevel.Release,
platform: platform,
//platform: platform,
specificDiagnosticOptions: new Dictionary<string, ReportDiagnostic>
{
{ "CS1701", ReportDiagnostic.Suppress }
@ -358,3 +379,4 @@ public static class scr
}
}
#endif

13
ser/XmlSer_Core.cs
View File

@ -121,7 +121,9 @@ public partial class ObjectHandler : ITypeHandler
}
private (object? obj, long id) GetOrCreateInstance( XmlSer xml, XmlElement elem, Type type, object? existing )
static public (object? obj, long id) GetOrCreateInstance( XmlSer xml, XmlElement elem, Type type, object? existing )
{
long id = -1;
bool first = true;
@ -137,14 +139,7 @@ public partial class ObjectHandler : ITypeHandler
object? newObj = null;
try
{
if( type.GetConstructor( Type.EmptyTypes ) != null )
{
newObj = Activator.CreateInstance( type );
}
else
{
newObj = FormatterServices.GetUninitializedObject( type );
}
newObj = refl.CreateObject( type );
}
catch( Exception ex )
{

25
sharplib.sln
View File

@ -1,25 +0,0 @@
Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 17
VisualStudioVersion = 17.5.002.0
MinimumVisualStudioVersion = 10.0.40219.1
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "SharpLib", "SharpLib.csproj", "{CC1801F8-7270-47A2-AF73-CCE2900549A9}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Release|Any CPU = Release|Any CPU
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{CC1801F8-7270-47A2-AF73-CCE2900549A9}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{CC1801F8-7270-47A2-AF73-CCE2900549A9}.Debug|Any CPU.Build.0 = Debug|Any CPU
{CC1801F8-7270-47A2-AF73-CCE2900549A9}.Release|Any CPU.ActiveCfg = Release|Any CPU
{CC1801F8-7270-47A2-AF73-CCE2900549A9}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {E2F905DE-69F3-48F3-943C-B8544945C2E9}
EndGlobalSection
EndGlobal

25
sharplib_be.sln
View File

@ -1,25 +0,0 @@
Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 17
VisualStudioVersion = 17.5.002.0
MinimumVisualStudioVersion = 10.0.40219.1
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "SharpLib", "SharpLib.csproj", "{8D6C0DEE-23AB-41CA-99B0-4575ECBB41F6}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Release|Any CPU = Release|Any CPU
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{8D6C0DEE-23AB-41CA-99B0-4575ECBB41F6}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{8D6C0DEE-23AB-41CA-99B0-4575ECBB41F6}.Debug|Any CPU.Build.0 = Debug|Any CPU
{8D6C0DEE-23AB-41CA-99B0-4575ECBB41F6}.Release|Any CPU.ActiveCfg = Release|Any CPU
{8D6C0DEE-23AB-41CA-99B0-4575ECBB41F6}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {2C2EFBD0-C7A0-4D9C-B6CD-68E0112B6786}
EndGlobalSection
EndGlobal

375
srl/srl.Core.cs
View File

@ -1,375 +0,0 @@
using System;
using System.Collections;
using System.Collections.Generic;
using System.ComponentModel;
using System.Linq;
using System.Reflection;
using System.Text;
namespace srl;
// --- INTERFACES ---
public interface Driver
{
// Structural
void BeginScope( string name, Type type, int id );
void EndScope();
void BeginCollection( string name, int count );
void EndCollection();
// Data
void WriteAttr( string name, string value ); // Primitives & Compact Strings
void WriteRef( string name, int id ); // DAG/Cycle Reference
// Metadata Hook
void OnProp( MemberInfo member, string name );
}
public interface IInput
{
string? Value { get; }
bool IsLeaf { get; }
IInput? GetAttr( string name );
IInput? GetChild( string name );
}
// --- THE MODEL (Introspection) ---
public class TypePlan
{
public string Name;
public bool IsCollection;
public bool IsHybrid; // True if Type has a StringParser registered
public StringParser Parser; // The "Compact" converter
public List<PropPlan> Props = new();
}
public class PropPlan
{
public string Name;
public string MemberName;
public Type Type;
public bool IsSmart; // Primitive/Enum/String
public MemberInfo Info;
// Fast Accessors
public Func<object, object?> Getter;
public Action<object, object?> Setter;
}
public static class Model
{
private static Dictionary<Type, TypePlan> _cache = new();
public static TypePlan Get( Type t )
{
if( _cache.TryGetValue( t, out var plan ) )
return plan;
plan = new TypePlan { Name = t.Name };
// 1. Check for Custom Parsers (Hybrid Mode)
if( Parsers.TryGet( t, out var parser ) )
{
plan.IsHybrid = true;
plan.Parser = parser;
}
// 2. Check for Collections
if( t != typeof( string ) && typeof( IEnumerable ).IsAssignableFrom( t ) )
{
plan.IsCollection = true;
_cache[t] = plan;
return plan;
}
// 3. Scan Properties
foreach( var p in t.GetProperties( BindingFlags.Public | BindingFlags.Instance ) )
{
if( Attribute.IsDefined( p, typeof( IgnoreAttribute ) ) )
continue;
var prop = new PropPlan
{
Name = p.Name, // Default to PascalCase, Drivers can lower if needed
MemberName = p.Name,
Type = p.PropertyType,
Info = p,
Getter = ( o ) => p.GetValue( o ),
Setter = ( o, v ) => p.SetValue( o, v )
};
// Override name
var nameAttr = p.GetCustomAttribute<NameAttribute>();
if( nameAttr != null )
prop.Name = nameAttr.Name;
// Is "Smart" (Atomic)?
prop.IsSmart = prop.Type.IsPrimitive || prop.Type.IsEnum ||
prop.Type == typeof( string ) || prop.Type == typeof( Guid );
plan.Props.Add( prop );
}
_cache[t] = plan;
return plan;
}
}
// --- ATTRIBUTES ---
public class IgnoreAttribute : Attribute { }
public class NameAttribute : Attribute { public string Name; public NameAttribute( string n ) => Name = n; }
// --- UTILITIES ---
public struct StringParser
{
public Func<object, string> To;
public Func<string, object> From;
}
public static class Parsers
{
private static Dictionary<Type, StringParser> _registry = new();
public static void Register<T>( Func<T, string> to, Func<string, T> from )
{
_registry[typeof( T )] = new StringParser { To = o => to( (T)o ), From = s => from( s ) };
}
public static bool TryGet( Type t, out StringParser p )
{
if( _registry.TryGetValue( t, out p ) )
return true;
// Auto-Discovery could go here (Static Parse methods)
return false;
}
}
public static class Binder
{
// Handles "Cool.Rare" dot notation
public static void Apply( object root, string path, string value )
{
var current = root;
var parts = path.Split( '.' );
for( int i = 0; i < parts.Length; i++ )
{
var part = parts[i];
var isLast = i == parts.Length - 1;
var plan = Model.Get( current.GetType() );
// Case-Insensitive Match
var prop = plan.Props.Find( p => p.Name.Equals( part, StringComparison.OrdinalIgnoreCase ) );
if( prop == null )
return;
if( isLast )
{
var val = ParseUtils.Convert( value, prop.Type );
prop.Setter( current, val );
}
else
{
var next = prop.Getter( current );
if( next == null )
{
next = Activator.CreateInstance( prop.Type );
prop.Setter( current, next );
}
current = next;
}
}
}
}
public static class ParseUtils
{
public static object Convert( string raw, Type t )
{
if( t == typeof( string ) )
return raw;
if( t == typeof( int ) )
return int.Parse( raw );
if( t == typeof( float ) )
return float.Parse( raw.Replace( "f", "" ) );
if( t == typeof( bool ) )
return bool.Parse( raw );
if( t.IsEnum )
return Enum.Parse( t, raw );
// Fallback to TypeConverter
var cv = TypeDescriptor.GetConverter( t );
if( cv != null && cv.CanConvertFrom( typeof( string ) ) )
return cv.ConvertFrom( raw );
return null;
}
public static void CopyFields( object src, object dst )
{
foreach( var p in src.GetType().GetProperties() )
if( p.CanRead && p.CanWrite )
p.SetValue( dst, p.GetValue( src ) );
}
}
// --- THE ENGINE (Walker & Loader) ---
public static class Walker
{
public class Context
{
private Dictionary<object, int> _seen = new( ReferenceEqualityComparer.Instance );
private int _nextId = 1;
public (int, bool) GetId( object o )
{
if( _seen.TryGetValue( o, out var id ) )
return (id, false);
_seen[o] = _nextId;
return (_nextId++, true);
}
}
public static void Serialize( object root, Driver d )
{
if( root == null )
return;
SerializeRecursive( root, d, new Context(), "root", null );
}
private static void SerializeRecursive( object obj, Driver d, Context ctx, string name, MemberInfo? member )
{
if( obj == null )
return;
Type type = obj.GetType();
var plan = Model.Get( type );
// STRATEGY 1: COMPACT (Hybrid Parser)
if( plan.IsHybrid )
{
if( member != null )
d.OnProp( member, name );
d.WriteAttr( name, plan.Parser.To( obj ) );
return;
}
// STRATEGY 2: DAG CHECK
bool isRef = !type.IsValueType && type != typeof( string );
if( isRef )
{
var (id, isNew) = ctx.GetId( obj );
if( !isNew )
{ d.WriteRef( name, id ); return; }
if( member != null )
d.OnProp( member, name );
d.BeginScope( name, type, id );
}
else
{
if( member != null )
d.OnProp( member, name );
d.BeginScope( name, type, 0 );
}
// STRATEGY 3: COLLECTIONS
if( plan.IsCollection )
{
var list = (IEnumerable)obj;
int count = 0; // Simple count (could optimize for ICollection)
foreach( var _ in list )
count++;
d.BeginCollection( name, count );
foreach( var item in list )
SerializeRecursive( item, d, ctx, "item", null );
d.EndCollection();
}
else
{
// STRATEGY 4: STANDARD OBJECT
foreach( var prop in plan.Props )
{
var val = prop.Getter( obj );
if( prop.IsSmart )
{
d.OnProp( prop.Info, prop.Name );
d.WriteAttr( prop.Name, val?.ToString() ?? "" );
}
else
{
if( val != null )
SerializeRecursive( val, d, ctx, prop.Name, prop.Info );
}
}
}
d.EndScope();
}
}
public static class Loader
{
public static void Load( object target, IInput input )
{
if( target == null || input == null )
return;
var plan = Model.Get( target.GetType() );
// 1. HYBRID PARSE (Compact String)
// If we have a parser AND input is just a value "1,1"
if( plan.IsHybrid && input.IsLeaf && !string.IsNullOrWhiteSpace( input.Value ) )
{
try
{
var newObj = plan.Parser.From( input.Value );
ParseUtils.CopyFields( newObj, target );
return;
}
catch { }
}
// 2. STRUCTURAL MAP
foreach( var prop in plan.Props )
{
// Look for Attribute OR Child Element
var sub = input.GetAttr( prop.Name ) ?? input.GetChild( prop.Name );
// Look for Dot Notation (e.g. "Pos.X") in attributes
// (Note: This simple loop doesn't scan ALL attrs for dots,
// it relies on the caller or specific recursive logic.
// For full dot support on root, we need to iterate input attributes if possible.
// But 'Binder' below handles it if we pass the specific attr key).
if( sub != null )
{
if( prop.IsSmart )
{
if( sub.Value != null )
prop.Setter( target, ParseUtils.Convert( sub.Value, prop.Type ) );
}
else
{
var child = prop.Getter( target );
if( child == null )
{
child = Activator.CreateInstance( prop.Type );
prop.Setter( target, child );
}
Load( child, sub );
}
}
}
}
// Helper to scan all attributes on an element for "Cool.Rare" patterns
public static void LoadDotNotations( object target, IEnumerable<(string k, string v)> attrs )
{
foreach( var (k, v) in attrs )
{
if( k.Contains( '.' ) )
Binder.Apply( target, k, v );
}
}
}

47
srl/srl.Debug.cs
View File

@ -1,47 +0,0 @@
using System;
using System.Reflection;
using System.Text;
namespace srl.Debug;
public class DebugDriver : Driver
{
private StringBuilder _sb = new();
private int _indent = 0;
public override string ToString() => _sb.ToString();
private void Line( string s ) => _sb.AppendLine( new string( ' ', _indent * 2 ) + s );
public void BeginScope( string name, Type type, int id )
{
var refStr = id > 0 ? $" #{id}" : "";
Line( $"[{name}] <{type.Name}>{refStr}" );
_indent++;
}
public void EndScope() => _indent--;
public void BeginCollection( string name, int count )
{
Line( $"[{name}] (Count: {count})" );
_indent++;
}
public void EndCollection() => _indent--;
public void WriteAttr( string name, string value )
{
Line( $"{name} = {value}" );
}
public void WriteRef( string name, int id )
{
Line( $"{name} -> See #{id}" );
}
public void OnProp( MemberInfo m, string name )
{
// Could log attributes here, e.g. [Tooltip]
}
}

131
srl/srl.Xml.cs
View File

@ -1,131 +0,0 @@
using System;
using System.Collections.Generic;
using System.Xml.Linq;
using System.Reflection;
namespace srl.Xml;
// --- INPUT ADAPTER ---
public class XmlAdapter : IInput
{
private XElement _e;
private XAttribute _a;
public XmlAdapter( XElement e ) => _e = e;
public XmlAdapter( XAttribute a ) => _a = a;
public string? Value => _e?.Value ?? _a?.Value;
public bool IsLeaf => _a != null || ( _e != null && !_e.HasElements );
public IInput? GetAttr( string name )
{
if( _e == null )
return null;
// Case-Insensitive search
var a = _e.Attribute( name ) ?? _e.Attribute( name.ToLower() );
return a != null ? new XmlAdapter( a ) : null;
}
public IInput? GetChild( string name )
{
if( _e == null )
return null;
var c = _e.Element( name ) ?? _e.Element( name.ToLower() );
return c != null ? new XmlAdapter( c ) : null;
}
// Extra helper for Dot Notation scanning
public IEnumerable<(string, string)> GetAllAttrs()
{
if( _e == null )
yield break;
foreach( var a in _e.Attributes() )
yield return (a.Name.LocalName, a.Value);
}
}
// --- OUTPUT DRIVER ---
public class XmlDriver : Driver
{
private Stack<XElement> _stack = new();
private XDocument _doc;
public XDocument Document => _doc;
public XmlDriver()
{
_doc = new XDocument();
}
public void BeginScope( string name, Type type, int id )
{
var el = new XElement( name );
// Polymorphism Metadata (if needed, e.g. <Item type="Sword">)
// el.Add(new XAttribute("_type", type.Name));
if( id > 0 )
el.Add( new XAttribute( "_id", id ) ); // DAG ID
if( _stack.Count > 0 )
_stack.Peek().Add( el );
else
_doc.Add( el );
_stack.Push( el );
}
public void EndScope() => _stack.Pop();
public void BeginCollection( string name, int count )
{
// XML doesn't strictly need array wrappers, but it helps structure
// We use the same BeginScope logic effectively
var el = new XElement( name, new XAttribute( "_count", count ) );
if( _stack.Count > 0 )
_stack.Peek().Add( el );
_stack.Push( el );
}
public void EndCollection() => _stack.Pop();
public void WriteAttr( string name, string value )
{
if( _stack.Count == 0 )
return;
_stack.Peek().Add( new XAttribute( name, value ) );
}
public void WriteRef( string name, int id )
{
var el = new XElement( name, new XAttribute( "_ref", id ) );
_stack.Peek().Add( el );
}
public void OnProp( MemberInfo m, string name ) { /* Optional: Write tooltips/comments */ }
}
// --- FACADE ---
public static class XmlSerializer
{
public static string Serialize( object obj )
{
var driver = new XmlDriver();
srl.Walker.Serialize( obj, driver );
return driver.Document.ToString();
}
public static void Deserialize( object root, string xml )
{
var doc = XDocument.Parse( xml );
var adapter = new XmlAdapter( doc.Root );
// 1. Standard Load
srl.Loader.Load( root, adapter );
// 2. Dot Notation Pass (MyBag Cool.Rare="Changed")
srl.Loader.LoadDotNotations( root, adapter.GetAllAttrs() );
}
}

16
task/Task.cs
View File

@ -1,16 +0,0 @@
using System.Threading.Tasks;
using System.Threading;
using System.Diagnostics;
namespace lib;
static public class Task
{
}

125
tests/Tests.cs
View File

@ -1,125 +0,0 @@
using System.Diagnostics;
using System.IO;
namespace test;
public record class SimpleImmutable( string Name, int Age ) : io.Timed<SimpleImmutable>;
static public class XmlFormatter2
{
public class ClassWithProperties
{
public string ActualProperty { get; set; } = "test_ActualProperty_set_inline";
public string ActualProperty_NotSerialized { get; set; } = "ActualProperty_NotSerialized";
}
[ser.Ser( Types = ser.Types.Implied )]
public partial class ClassContainsClassWithProp
{
[ser.Do]
public bool doBool = true;
[ser.ChildPropsAttribute( "ActualProperty" )]
public ClassWithProperties propHolder = new();
public string doNotSerialize = "test_do_not_serialize";
}
[ser.Ser]
public class ClassHasFields
{
public ClassContainsClassWithProp prop = new();
};
public static void Serialization()
{
lib.XmlFormatter2Cfg cfg = new()
{
};
ClassHasFields classHasFields = new()
{
prop = new()
{
propHolder = new()
{
ActualProperty = "ActualProperty_set_in_cons"
}
}
};
Debug.Assert( classHasFields.prop.propHolder.ActualProperty == "ActualProperty_set_in_cons" );
var memStream = new MemoryStream();
{
var xml = new lib.XmlFormatter2( cfg );
xml.Serialize( memStream, classHasFields );
}
memStream.Position = 0;
var strXml = System.Text.Encoding.UTF8.GetString( memStream.ToArray() );
var badXml = "<root>\n <prop doBool=\"True\" />\n</root>";
/*
<root _.t="test.XmlFormatter2+ClassHasFields" _.version.="2">
<prop doBool="True">
<propHolder ActualProperty="ActualProperty_set_in_cons" ActualProperty_NotSerialized="ActualProperty_NotSerialized" ActualProperty="ActualProperty_set_in_cons" />
</prop>
</root>*/
Debug.Assert( strXml != badXml );
memStream.Position = 0;
var classHasFields2 = new ClassHasFields();
classHasFields2.prop.propHolder.ActualProperty_NotSerialized = "ActualProperty_NotSerialized_set_in_test_01";
Debug.Assert( classHasFields2.prop.propHolder.ActualProperty == "test_ActualProperty_set_inline" );
Debug.Assert( classHasFields2.prop.propHolder.ActualProperty_NotSerialized == "ActualProperty_NotSerialized" );
{
var xml = new lib.XmlFormatter2( cfg );
classHasFields2 = xml.Deserialize<ClassHasFields>( memStream );
}
Debug.Assert( classHasFields2.prop.propHolder.ActualProperty == "ActualProperty_set_in_cons" );
Debug.Assert( classHasFields2.prop.propHolder.ActualProperty_NotSerialized == "ActualProperty_NotSerialized_set_in_test_01" );
memStream.Position = 0;
var classHasFields3 = new ClassHasFields();
{
var xml = new lib.XmlFormatter2( cfg );
xml.DeserializeInto( memStream, classHasFields3 );
}
Debug.Assert( classHasFields3.prop.propHolder.ActualProperty == "ActualProperty_set_in_cons" );
Debug.Assert( classHasFields3.prop.propHolder.ActualProperty_NotSerialized == "ActualProperty_NotSerialized_set_in_test_01" );
}
}

2
time/Time.cs
View File

@ -3,6 +3,8 @@
using System;
namespace time;

1
util/Exec.cs
View File

@ -1,6 +1,7 @@
using System;
using System.Diagnostics;
namespace lib;