Sunday, July 5, 2015

C# in a Nutshell Chapter 15 - Streams and I/O

The .NET stream architecture centers on three concepts: backing stores, decorators, and adapters.

Backing store streams
These are hard-wired to a particular type of backing store, such as FileStream or NetworkStream

Decorator streams
These feed off another stream, transforming the data in some way, such as DeflateStream or CryptoStream

Both backing store and decorator streams deal exclusively in bytes. Although this is flexible and efficient, applications often work at higher levels such as text or XML. Adapters bridge this gap by wrapping a stream in a class with specialized methods typed to a particular format. For example, a text reader exposes a ReadLine method; an XML writer exposes a WriteAttributes method.

An adapter wraps a stream, just like a decorator. Unlike a decorator, however, an adapter is not itself a stream; it typically hides the byte-oriented methods completely.

To summarize, backing store streams provide the raw data; decorator streams provide transparent binary transformations such as encryption; adapters offer typed methods for dealing in higher-level types such as strings and XML. To compose a chain, you simply pass one object into another’s constructor.

Stream Class

using System;
using System.IO;
class Program
static void Main()
// Create a file called test.txt in the current directory:
using (Stream s = new FileStream ("test.txt", FileMode.Create))
Console.WriteLine (s.CanRead);
// True
Console.WriteLine (s.CanWrite);
// True
Console.WriteLine (s.CanSeek);
// True

s.WriteByte (101);
s.WriteByte (102);
byte[] block = { 1, 2, 3, 4, 5 };
s.Write (block, 0, block.Length); // Write block of 5 bytes
Console.WriteLine (s.Length);
Console.WriteLine (s.Position);
s.Position = 0; // 7
// 7
// Move back to the start
Console.WriteLine (s.ReadByte());
Console.WriteLine (s.ReadByte()); // 101
// 102
// Read from the stream back into the block array:
Console.WriteLine (s.Read (block, 0, block.Length)); // 5
// Assuming the last Read returned 5, we'll be at
// the end of the file, so Read will now return 0:
Console.WriteLine (s.Read (block, 0, block.Length)); // 0

Reading and Writing

A stream may support reading, writing, or both. If CanWrite returns false , the stream is read-only; if CanRead returns false , the stream is write-only.

With Read , you can be certain you’ve reached the end of the stream only when the method returns 0 . So, if you have a 1,000 byte stream, the following code may fail to read it all into memory:

// Assuming s is a stream:
byte[] data = new byte [1000];
s.Read (data, 0, data.Length);

The Read method could read anywhere from 1 to 1,000 bytes, leaving the balance of the stream unread.

Here’s the correct way to read a 1,000-byte stream:

byte[] data = new byte [1000];
// bytesRead will always end up at 1000, unless the stream is itself smaller in length:
int bytesRead = 0;
int chunkSize = 1;
while (bytesRead < data.Length && chunkSize > 0)
bytesRead += chunkSize = s.Read (data, bytesRead, data.Length - bytesRead);

Fortunately, the BinaryReader type provides a simpler way to achieve the same result:

byte[] data = new BinaryReader (s).ReadBytes (1000);

If the stream is less than 1,000 bytes long, the byte array returned reflects the actual stream size. If the stream is seekable, you can read its entire contents by replacing 1000 with (int)s.Length .


A stream is seekable if CanSeek returns true . With a seekable stream (such as a file stream), you can query or modify its Length (by calling SetLength ), and at any time change the Position at which you’re reading or writing. The Position property is relative to the beginning of the stream; the Seek method, however, allows you to move relative to the current position or the end of the stream.

With a nonseekable stream (such as an encryption stream), the only way to determine its length is to read it right through. Furthermore, if you need to reread a previous section, you must close the stream and start afresh with a new one.

Closing and Flush

Streams must be disposed after use to release underlying resources such as file and socket handles. A simple way to guarantee this is by instantiating streams within using blocks.

In general, streams follow standard disposal semantics:

  • Dispose and Close are identical in function.
  • Disposing or closing a stream repeatedly causes no error.
Closing a decorator stream closes both the decorator and its backing store stream. With a chain of decorators, closing the outermost decorator (at the head of the chain) closes the whole lot.

Some streams internally buffer data to and from the backing store to lessen round tripping and so improve performance (file streams are a good example of this). This means data you write to a stream may not hit the backing store immediately; it can be delayed as the buffer fills up. The Flush method forces any internally buffered data to be written immediately. Flush is called automatically when a stream is closed, so you never need to do the following: s.Flush(); s.Close();


A stream supports read and write timeouts if CanTimeout returns true . Network streams support timeouts; file and memory streams do not. For streams that support timeouts, the ReadTimeout and WriteTimeout properties determine the desired timeout in milliseconds, where 0 means no timeout. The Read and Write methods indicate that a timeout has occurred by throwing an exception.

Backing Store Streams


The simplest way to instantiate a FileStream is to use one of the following static methods on the File class:

FileStream fs1 = File.OpenRead ("readme.bin");  // Read-only
FileStream fs2 = File.OpenWrite (@"c:\temp\writeme.tmp");  // Write-only
FileStream fs3 = File.Create (@"c:\temp\writeme.tmp"); // Read/write

OpenWrite and Create differ in behavior if the file already exists. Create truncates any existing content; OpenWrite leaves existing content intact with the stream positioned at zero. If you write fewer bytes than were previously in the file, OpenWrite leaves you with a mixture of old and new content.

Instantiate a FileStream is also possible. The following opens an existing file for read/write access without overwriting it:

var fs = new FileStream ("readwrite.tmp", FileMode.Open);

File Class

The following static methods read an entire file into memory in one step:
• File.ReadAllText (returns a string)
• File.ReadAllLines (returns an array of strings)
• File.ReadAllBytes (returns a byte array)
The following static methods write an entire file in one step:
• File.WriteAllText
• File.WriteAllLines
• File.WriteAllBytes
• File.AppendAllText (great for appending to a log file)

There’s also a static method called File.ReadLines : this is like ReadAllLines except that it returns a lazily-evaluated IEnumerable<string> . This is more efficient because it doesn’t load the entire file into memory at once. LINQ is ideal for consuming the results: the following calculates the number of lines greater than 80 characters in length:
int longLines = File.ReadLines ("filePath").Count (l => l.Length > 80);

File Mode


Closing and flushing a MemoryStream is optional. If you close a MemoryStream , you can no longer read or write to it, but you are still permitted to call ToArray to obtain the underlying data.
Flush does absolutely nothing on a memory stream.


PipeStream was introduced in Framework 3.5. It provides a simple means by which one process can communicate with another through the Windows pipes protocol.
There are two kinds of pipe:

  1. Anonymous pipe: Allows one-way communication between a parent and child process on the same computer.
  2. Named pipe: Allows two-way communication between arbitrary processes on the same computer—or different computers across a Windows network.

PipeStream is an abstract class with four concrete subtypes. Two are used for anonymous pipes and the other two for named pipes:

  1. AnonymousPipeServerStream and AnonymousPipeClientStream
  2. NamedPipeServerStream and NamedPipeClientStream


BufferedStream decorates, or wraps, another stream with buffering capability.
Buffering improves performance by reducing round trips to the backing store. Here’s how we wrap a FileStream in a 20 KB BufferedStream :

// Write 100K to a file:
File.WriteAllBytes ("myFile.bin", new byte [100000]);
using (FileStream fs = File.OpenRead ("myFile.bin"))
using (BufferedStream bs = new BufferedStream (fs, 20000))
Console.WriteLine (fs.Position);
// 20000

In this example, the underlying stream advances 20,000 bytes after reading just 1 byte, thanks to the read-ahead buffering. We could call ReadByte another 19,999 times before the FileStream would be hit again.

Coupling a BufferedStream to a FileStream , as in this example, is of limited value because FileStream already has built-in buffering. Its only use might be in enlarging the buffer on an already constructed FileStream .
Closing a BufferedStream automatically closes the underlying backing store stream.

Stream Adapters

A Stream deals only in bytes; to read or write data types such as strings, integers, or XML elements, you must plug in an adapter.

Text Adapters

TextReader and TextWriter are the abstract base classes for adapters that deal exclusively with characters and strings.

using (FileStream fs = File.Create ("test.txt"))
using (TextWriter writer = new StreamWriter (fs))
writer.WriteLine ("Line1");
writer.WriteLine ("Line2");
using (FileStream fs = File.OpenRead ("test.txt"))
using (TextReader reader = new StreamReader (fs))
Console.WriteLine (reader.ReadLine()); // Line1
Console.WriteLine (reader.ReadLine()); // Line2

Because text adapters are so often coupled with files, the File class provides the static methods CreateText , AppendText , and OpenText to shortcut the process:

using (TextWriter writer = File.CreateText ("test.txt"))
writer.WriteLine ("Line1");
writer.WriteLine ("Line2");
using (TextWriter writer = File.AppendText ("test.txt"))
writer.WriteLine ("Line3");
using (TextReader reader = File.OpenText ("test.txt"))
while (reader.Peek() > −1)
Console.WriteLine (reader.ReadLine());

This also illustrates how to test for the end of a file (viz. reader.Peek() ). Another option is to read until reader.ReadLine returns null.

You can also read and write other types such as integers, but because TextWriter invokes ToString on your type, you must parse a string when reading it back:

using (TextWriter w = File.CreateText ("data.txt"))
w.WriteLine (123);
// Writes "123"
w.WriteLine (true);
// Writes the word "true"
using (TextReader r = File.OpenText ("data.txt"))
int myInt = int.Parse (r.ReadLine());
// myInt == 123
bool yes = bool.Parse (r.ReadLine());
// yes == true

Character encodings

TextReader and TextWriter are by themselves just abstract classes with no connection to a stream or backing store. The StreamReader and StreamWriter types, however, are connected to an underlying byte-oriented stream, so they must convert between characters and bytes. They do so through an Encoding class from the System.Text namespace, which you choose when constructing the StreamReader or StreamWriter . If you choose none, the default UTF-8 encoding is used.

StringReader and StringWriter

The StringReader and StringWriter adapters don’t wrap a stream at all; instead, they use a string or StringBuilder as the underlying data source. This means no byte translation is required—in fact, the classes do nothing you couldn’t easily achieve with a string or StringBuilder coupled with an index variable. Their advantage, though, is that they share a base class with StreamReader / StreamWriter . For instance, suppose we have a string containing XML and want to parse it with an XmlReader .
The XmlReader.Create method accepts one of the following:

  1. A URI
  2. A Stream
  3. A TextReader

So, how do we XML-parse our string? Because StringReader is a subclass of TextReader , we’re in luck. We can instantiate and pass in a StringReader as follows:

XmlReader r = XmlReader.Create (new StringReader (myString));

Binary Adapters

BinaryReader and BinaryWriter read and write native data types: bool , byte , char ,
decimal , float , double , short , int , long , sbyte , ushort , uint , and ulong , as well as

string s and arrays of the primitive data types.

BinaryReader can also read into byte arrays. The following reads the entire contents
of a seekable stream:

byte[] data = new BinaryReader (s).ReadBytes ((int) s.Length);

This is more convenient than reading directly from a stream, because it doesn't require a loop to ensure that all data has been read.

Compression Streams

Two general-purpose compression streams are provided in the System.IO.Compression namespace: DeflateStream and GZipStream.

DeflateStream and GZipStream are decorators; they compress or decompress data from another stream that you supply in construction. In the following example, we compress and decompress a series of bytes, using a FileStream as the backing store:

using (Stream s = File.OpenRead ("compressed.bin"))
using (Stream ds = new DeflateStream (s, CompressionMode.Decompress))
for (byte i = 0; i < 100; i++)
Console.WriteLine (ds.ReadByte());
// Writes 0 to 99

Even with the smaller of the two algorithms, the compressed file is 241 bytes long: more than double the original! Compression works poorly with “dense,” nonrepetitive binary filesdata!

In the next example, we compress and decompress a text stream composed of 1,000 words chosen randomly from a small sentence. This also demonstrates chaining a backing store stream, a decorator
stream, and an adapter (as depicted at the start of the chapter in Figure 15-1), and the use asynchronous methods:

string[] words = "The quick brown fox jumps over the lazy dog".Split();
Random rand = new Random();
using (Stream s = File.Create ("compressed.bin"))
using (Stream ds = new DeflateStream (s, CompressionMode.Compress))
using (TextWriter w = new StreamWriter (ds))
for (int i = 0; i < 1000; i++)
await w.WriteAsync (words [rand.Next (words.Length)] + " ");
Console.WriteLine (new FileInfo ("compressed.bin").Length);
// 1073
using (Stream s = File.OpenRead ("compressed.bin"))
using (Stream ds = new DeflateStream (s, CompressionMode.Decompress))
using (TextReader r = new StreamReader (ds))
Console.Write (await r.ReadToEndAsync()); // Output;

In this case, DeflateStream compresses efficiently to 1,073 bytes—slightly more than 1 byte per word.

Compressing in memory

Sometimes you need to compress entirely in memory. Here’s how to use a Memory Stream for this purpose:

byte[] data = new byte[1000];
// We can expect a good compression
// ratio from an empty array!
var ms = new MemoryStream();
using (Stream ds = new DeflateStream (ms, CompressionMode.Compress))
ds.Write (data, 0, data.Length);
byte[] compressed = ms.ToArray();
Console.WriteLine (compressed.Length);
// 113
// Decompress back to the data array:
ms = new MemoryStream (compressed);
using (Stream ds = new DeflateStream (ms, CompressionMode.Decompress))
for (int i = 0; i < 1000; i += ds.Read (data, i, 1000 - i));

The using statement around the DeflateStream closes it in a textbook fashion, flushing any unwritten buffers in the process. This also closes the MemoryStream it wraps —meaning we must then call ToArray to extract its data.

Working with Zip Files

new feature in Framework 4.5 - ZipArchive and ZipFile classes

ZipFile is a static helper class for ZipArchive;

ZipFile ’s CreateFromDirectory method adds all the files in a specified directory into a zip file:
ZipFile.CreateFromDirectory (@"d:\MyFolder", @"d:\");

whereas ExtractToDirectory does the opposite and extracts a zip file to a directory:
ZipFile.ExtractToDirectory (@"d:\", @"d:\MyFolder");

File and Directory Operations

FileInfo offers an easier way to change a file’s read-only flag:
new FileInfo (@"c:\temp\test.txt").IsReadOnly = false;

Here are all the members of the FileAttribute enum that GetAttributes returns:

Archive, Compressed, Device, Directory, Encrypted, Hidden, Normal, NotContentIndexed, Offline, ReadOnly, ReparsePoint, SparseFile, System, Temporary

File security

The GetAccessControl and SetAccessControl methods allow you to query and change the operating system permissions assigned to users and roles via a FileSecurity object (namespace System.Security.AccessControl ). You can also pass a FileSecurity object to a FileStream ’s constructor to specify permissions when creating a new file.

In this example, we list a file’s existing permissions, and then assign execution permission to the “Users” group:

FileSecurity sec = File.GetAccessControl (@"d:\test.txt");
AuthorizationRuleCollection rules = sec.GetAccessRules (true, true,
typeof (NTAccount));
foreach (FileSystemAccessRule rule in rules)
Console.WriteLine (rule.AccessControlType); // Allow or Deny
Console.WriteLine (rule.FileSystemRights); // e.g., FullControl
Console.WriteLine (rule.IdentityReference.Value); // e.g., MyDomain/Joe
var sid = new SecurityIdentifier (WellKnownSidType.BuiltinUsersSid, null);
string usersAccount = sid.Translate (typeof (NTAccount)).ToString();
FileSystemAccessRule newRule = new FileSystemAccessRule
(usersAccount, FileSystemRights.ExecuteFile, AccessControlType.Allow);
sec.AddAccessRule (newRule);
File.SetAccessControl (@"d:\test.txt", sec);

The Directory Class

he static Directory class provides a set of methods analogous to those in the File class—for checking whether a directory exists ( Exists ), moving a directory ( Move ), deleting a directory ( Delete ), getting/setting times of creation or last access, and getting/setting security permissions. Furthermore, Directory exposes the following static methods:

string GetCurrentDirectory ();
SetCurrentDirectory (string path);
DirectoryInfo CreateDirectory (string path);
DirectoryInfo GetParent
(string path);
GetDirectoryRoot (string path);
string[] GetLogicalDrives();
// The following methods all return full paths:
string[] GetFiles
(string path);
string[] GetDirectories
(string path);
string[] GetFileSystemEntries (string path);
IEnumerable<string> EnumerateFiles (string path);
IEnumerable<string> EnumerateDirectories (string path);
IEnumerable<string> EnumerateFileSystemEntries (string path);

FileInfo and DirectoryInfo

The static methods on File and Directory are convenient for executing a single file or directory operation. If you need to call a series of methods in a row, the FileInfo and DirectoryInfo classes provide an object model that makes the job easier.

FileInfo offers most of the File ’s static methods in instance form—with some additional properties such as Extension , Length , IsReadOnly , and Directory —for returning a DirectoryInfo object. For example:

FileInfo fi = new FileInfo (@"c:\temp\FileInfo.txt");
Console.WriteLine (fi.Exists); // false
using (TextWriter w = fi.CreateText())
w.Write ("Some text");
Console.WriteLine (fi.Exists);  // false (still)
Console.WriteLine (fi.Exists);  // true

(fi.Name); // FileInfo.txt
(fi.FullName);  //c:\temp\FileInfo.txt
(fi.DirectoryName); //c:\temp
(fi.Directory.Name); //temp
(fi.Extension); // .txt
(fi.Length); // 9

fi.Attributes ^= FileAttributes.Hidden; //(toggle hidden flag)
fi.IsReadOnly = true;
Console.WriteLine (fi.Attributes); // ReadOnly,Archive,Hidden,Encrypted
Console.WriteLine (fi.CreationTime);

fi.MoveTo (@"c:\temp\FileInfoX.txt");
DirectoryInfo di = fi.Directory;
Console.WriteLine (di.Name);   // temp
Console.WriteLine (di.FullName);  // c:\temp
Console.WriteLine (di.Parent.FullName); // c:\
di.CreateSubdirectory ("SubFolder");

Here’s how to use DirectoryInfo to enumerate files and subdirectories:

DirectoryInfo di = new DirectoryInfo (@"e:\photos");
foreach (FileInfo fi in di.GetFiles ("*.jpg"))
Console.WriteLine (fi.Name);
foreach (DirectoryInfo subDir in di.GetDirectories())
Console.WriteLine (subDir.FullName);

No comments:

Post a Comment