DotnetStreams

Read text from STDIN regardless of the source.

Description

This library lets you read text input from STDIN that is entered from the console, piped to the application on the command line without any knowledge of the method. This is useful because you would normally not know if the stream has ended or not and have to check for it differently depending on the source.

Consuming the Library

You can use the library in several ways:

• By copying the code to your source repo and adding a project reference to the DotnetStreams project.
• By adding a NuGet folder resource to your NuGet sources configuration which points to this project.
• By adding a NuGet reference to the DotnetStreams project in the repo at nuget.pillidar.com.

How To Configure NuGet Sources

In Visual Studio 2022, right-click the Dependencies node in the Solution Explorer for your project. Click Manage NuGet Packages. In the upper-right corner, click the cog icon next to the Package Source dropdown.

Adding a NuGet Folder Resource

Add a new Package Source. | | | | --- | --- | | Name | DotnetStreams Folder | | Source | C:\dev\DotnetStreams\source\DotnetStreams\bin\Release |

Adding a NuGet Repo Reference to nuget.pillidar.com

Add a new Package Source. | | | | --- | --- | | Name | pillidar.com | | Source | https://nuget.pillidar.com/v3/index.json |

Adding a NuGet repo at nuget.pillidar.com

Usage Instructions

The Consuming Application

Your console app can use DotnetStreams to enable usage such as the following examples, using a single approach in code regardless of the source of your input:

echo "This is a test." | myconsoleapp.exe

type myfile.txt | myconsoleapp.exe

myconsoleapp.exe --filename myfile.txt

myconsoleapp.exe "line1" "line2" "line3"

c:> myconsoleapp.exe
line 1
line 2

line 3
^Z

The above scenarios can all be handled in a single application in same way using an interface. It may look something like this:

ITextSource source;
if (args[0].Equals("--filename")) 
	source = new FileTextSource(args[1]);
else if (args.Length > 0)
	source = new ListTextSource(args);
else
	source = new StdInTextSource();

ITextSource

ITextSource is the abstraction of the available kinds of inputs.

Implementations of ITextSource have Open() and Close() methods. They must be called for the implementations to operate properly. The Close() method should always be in a finally block.

Example:

ITextSource source = ... ;
source.Open();
try
{
	IEnumerable<string> lines = source.ReadAll(); 
}
finally
{
	source.Close();
}

IOutputTarget

Like ITextSource, implementations of IOutputTarget have Open() and Close() methods. They must be called for the implementations to operate properly. The Close() method should always be in a finally block.

IOutputTarget outputTarget = ... ;
outputTarget.Open();
try
{
	outputTarget.Output("Hello World");
}
finally
{
	outputTarget.Close();
}

Several Sources to Read

ITextSource

As described above, this is the interface that all text readers implement. It abstracts the source and the EOF behavior for uniform consumption of text sources. It has Open() and Close() fixture methods for implementations that need them.

StdInTextSource

This reads lines of text from the StdIn stream. This means that data can come from the Keyboard or can be piped in through the command line using the | pipe character using a type command to provide the contents of a file or using write-output (PowerShell) or echo (Cmd) to provide a literal text string from the command line. StdIn can also be redevined outside of the application in interesting ways.

FileTextSource

This reads lines of text from a text file on disk. You will need to provide your own mechanism for determining when to use it and what the filename will be. For example, in this document most examples demonstrate doing this by passing the filename as a command-line parameter which I think seems quite obvious.

ListTextSource

This wraps an IEnumerable<string>. This means it can also wrap a list. That, in turn, means you can wrap another ITextSource. This was written for experimentation and unit testing but may have other creative uses, such as transforming input sources by sorting the data, for example.

Several Targets to Write

IOutputTarget

As described above, this is a simple interface that wraps an output destination. It has Open() and Close() methods to match the ITextSource and an Output() method to do the writing. The Output() method does NOT insert or append newlines or other line endings. Since you probably already have them in your strings from their original ITextSource or other processing, you won't usually need them. If your strings do not have line endings but need them, you must concatenate them yourself.

ListOutputTarget

This wraps an IList<string> and will store data to it when Output() is called. This is useful for unit testing or other creative workarounds.

Example:

// Writes to the console.
List<string> expected = ["1", "2", "3"];
List<string> actual = [];
IOutputTarget outputWriter = new ListOutputTarget(actual);

for(int j = 1; j <= 3; j++)
	outputWriter(j.ToString());

actual.ShouldBe(expected);

ConsoleOutputTarget

This writes data to the StdOut device when Output() is called.

Example:

// Writes to the console.
IOutputTarget consoleTarget = new ConsoleOutputTarget();
consoleTarget.Output("Super Califragilistic Expiyallydocious!");

ProcessedConsoleOutputTarget

This class inherits ConsoleOutputTarget but adds a custom function to transform the text befor being output. This is very similar to the AnonOutputTarget except that it is specific to console output and it only provides projection / transforms, not filtering.

Example:

// Writes to the console.
IOutputTarget formattedTarget = new ProcessedConsoleOutputTarget(static s => 
{
	// This class does not directly perform its writes.
	// That means it can transform the input but cannot filter it.
	int maxLen = 15;
	string truncatedString = s.Substring(0, Math.Min(s.Length, maxLen))
	return $"[{DateTime.Now}] {truncatedString}";
};

formattedTarget.Output("Super Califragilistic Expiyallydocious!");

AnonOutputTarget

This will take an Action<string> parameter in its constructor to instruct it how to behave when Output() is called. This implementation can easily be used to fulfill any output need. Other obvious OutputTarget types (e.g. TextFileOutputTarget) have not been implemented because this method is simpler to use than those classes would be and it gives you more control than you would have with those classes. For example, you can control the access parameters when opening a text file.

Example:

// Writes to the console.
IOutputTarget errorLogger = new AnonOutputTarget(static s => 
{
	// Unlike ProcessedConsoleOutputTarget, the AnonOutputTarget
	// class is responsible for its own writes. This means it can 
	// filter and transform its output.
	if (s.ToLower().Contains("error"))
	{
		int maxLen = 15;
		string truncatedString = s.Substring(0, Math.Min(s.Length, maxLen))
		string logLine = $"[{DateTime.Now}] {truncatedString}";
		Console.WriteLine(logLine);
	}
};

errorLogger.Output("An error occurred while trying to connect to ...");

Code Usage

static void Main(string[] args)
{
	ITextSource textSource;
	IOutputTarget textTarget = new ConsoleOutputTarget();

	if (args.Length > 0)
		textSource = new FileTextSource(args[0]);
	else
		textSource = new StdInTextSource();

	Execute(
		textSource,
		textTarget);
}

public static void Execute(ITextSource source, IOutputTarget target)
{
	source.Open();
	target.Open();

	foreach (string line in source.ReadAll())
		target.Output(line);

	target.Close();
	source.Close();
}

Several Ways To Read

Iterator

You can read all lines using an iterator:

foreach(string line in source.ReadAll())
	// Do stuff with line here.

Callback

You can read all lines using a callback:

source.ReadAll(static line => /*do stuff with line here*/);

Input / Output Method Group

The action callback version of the ReadAll() method is compatible with the delegate for IOutputTarget.Output(). Therefore, you can pass an IOutputTarget.Output method directly as a parameter to the ReadAll() method without wrapping it in a lambda function.

ITextSource source = ... ;
IOutputTarget target = ... ;
source.Open();
target.Open();

// Notice .Output does not have parentheses and this is not using a lambda.  We are passing the function directly.
source.ReadAll(target.Output); 

target.Close();
source.Close();

One Line at a Time in a While Loop

You can read one line at a time in a loop, watching for EOF. Be careful to check for EOF in this specific way. The text source implementations are specifically engineered to behave with this logic:

EOF is FALSE after each read that returns data. EOF is TRUE after the first and subsequent reads where data is exhausted.

ITextSource source = ... ;
string line = source.Read();
while (!source.Eof())
{
	// Do stuff with line here.

	string line = source.Read();
}

One Line at a Time in a FOR Loop

source.Open();
target.Open();

for (string? line = source.Read(); !source.Eof(); line = source.Read())
{
	target.Output(line);
}

target.Close();
source.Close();

ListTextSource

This is a text source that wraps an in-memory List<string> object. It is useful for unit testing or for creative workarounds. This is also useful as if you want to use it to wrap input from other text sources, such as a FileTextSource when you want to do document-level operations, such as sorting all the lines, which require all lines to be available at once instead of one at a time in an iterator pattern as is provided by the text source itself. It takes an IEnumerable<string> as a constructor parameter, so it basically just wraps another collection as-is. Example:

ITextSource stdin = new StdInTextSource();
stdin.Open();
List<string> sortedStdIn = new List<string>(stdin.ReadAll());
sortedStdIn.Sort();
stdin.Close();

ITextSource listSource = new ListTextSource(sortedStdIn);
listSource.Open();
listSource.ReadAll(static line => { /*Do stuff with line here.*/ });
listSource.Close();

Why not IDisposable?

At its inception, there were no ITextSource or IOutputTarget implementations that had underlying IDisposable aspects. It wasn't until I added the FileTextSource that I encountered one.

Now that we have one, the intent is to unwind the Dispose() handling in that class, Modify ITextSource and IOutputTarget to inherit IDisposable, then modify all implementations to implement IDisposable. This will probably involve a base class to generalize the disposal code and also the ReadAll(Action<string> readAction) method implementation since it's repeated verbatim in all the ITextSource implementations.

This will be done in the next version.

Why not IEnumerable?

This is being considered. The primary concern is that the operations must be Open()d and Close()d. Along those lines, the Open() and Close() may be reconsidered as well and consolidated into the constructor and Dispose() methods, getting rid of them altogether. The IEnumerable would then be freely enumerated. I will experiment with this in the next few versions.

Why not Async?

Generally speaking, async operations should be used for I/O-bound or CPU-bound operations. These operations are not really CPU-bound as items can be iterated independently. They may be I/O-bound but that depends on the implementation. The nature of these operations generally puts them in a category of startup code, wherein you can't do much else until they have completed anyway.

I am still considering adding async versions but for now I am satisfied with synchronous operation.