The .NET stream architecture centers on three concepts: backing stores, decorators, and adapters.
These are hard-wired to a particular type of backing store, such as FileStream or NetworkStream
These feed off another stream, transforming the data in some way, such as DeflateStream or CryptoStream
Both backing store and decorator streams deal exclusively in bytes. Although this is flexible and efficient, applications often work at higher levels such as text or XML. Adapters
bridge this gap by wrapping a stream in a class with specialized methods typed to a particular format. For example, a text reader exposes a ReadLine method; an XML writer exposes a WriteAttributes method.
An adapter wraps a stream, just like a decorator. Unlike a decorator, however, an adapter is not itself a stream; it typically hides the byte-oriented methods completely.
backing store streams provide the raw data; decorator streams provide transparent binary transformations such as encryption; adapters offer typed methods for dealing in higher-level types such as strings and XML. To compose a chain, you simply pass one object into another’s constructor.
A stream may support reading, writing, or both. If CanWrite returns false , the stream is read-only; if CanRead returns false , the stream is write-only.
With Read , you can be certain you’ve reached the end of the stream only when the method returns 0 . So, if you have a 1,000 byte stream, the following code may fail to read it all into memory:
Fortunately, the BinaryReader type provides a simpler way to achieve the same result:
byte[] data = new BinaryReader (s).ReadBytes (1000);
If the stream is less than 1,000 bytes long, the byte array returned reflects the actual stream size. If the stream is seekable, you can read its entire contents by replacing 1000 with (int)s.Length .
A stream is seekable if CanSeek returns true . With a seekable stream (such as a file stream), you can query or modify its Length (by calling SetLength ), and at any time change the Position at which you’re reading or writing. The Position property is relative to the beginning of the stream; the Seek method, however, allows you to move relative to the current position or the end of the stream.
With a nonseekable stream (such as an encryption stream), the only way to determine its length is to read it right through. Furthermore, if you need to reread a previous section, you must close the stream and start afresh with a new one.
Closing a decorator stream closes both the decorator and its backing store stream. With a chain of decorators, closing the outermost decorator (at the head of the chain) closes the whole lot.
Some streams internally buffer data to and from the backing store to lessen round tripping and so improve performance (file streams are a good example of this). This means data you write to a stream may not hit the backing store immediately; it can be delayed as the buffer fills up. The Flush method forces any internally buffered data to be written immediately. Flush is called automatically when a stream is closed, so you never need to do the following: s.Flush(); s.Close();
File Class
The following static methods read an entire file into memory in one step:
• File.ReadAllText (returns a string)
• File.ReadAllLines (returns an array of strings)
• File.ReadAllBytes (returns a byte array)
The following static methods write an entire file in one step:
• File.WriteAllText
• File.WriteAllLines
• File.WriteAllBytes
• File.AppendAllText (great for appending to a log file)
There’s also a static method called File.ReadLines : this is like ReadAllLines except that it returns a lazily-evaluated IEnumerable<string> . This is more efficient because it doesn’t load the entire file into memory at once. LINQ is ideal for consuming the results: the following calculates the number of lines greater than 80 characters in length:
int longLines = File.ReadLines ("filePath").Count (l => l.Length > 80);
File Mode
MemoryStream
Closing and flushing a MemoryStream is optional. If you close a MemoryStream , you can no longer read or write to it, but you are still permitted to call ToArray to obtain the underlying data.
Flush does absolutely nothing on a memory stream.
PipeStream
PipeStream was introduced in Framework 3.5. It provides a simple means by which one process can communicate with another through the Windows pipes protocol.
There are two kinds of pipe:
- Anonymous pipe: Allows one-way communication between a parent and child process on the same computer.
- Named pipe: Allows two-way communication between arbitrary processes on the same computer—or different computers across a Windows network.
PipeStream is an abstract class with four concrete subtypes. Two are used for anonymous pipes and the other two for named pipes:
- AnonymousPipeServerStream and AnonymousPipeClientStream
- NamedPipeServerStream and NamedPipeClientStream
BufferedStream
BufferedStream decorates, or wraps, another stream with buffering capability.
Buffering improves performance by reducing round trips to the backing store. Here’s how we wrap a FileStream in a 20 KB BufferedStream :
// Write 100K to a file:
File.WriteAllBytes ("myFile.bin", new byte [100000]);
using (FileStream fs = File.OpenRead ("myFile.bin"))
using (BufferedStream bs = new BufferedStream (fs, 20000))
{
bs.ReadByte();
Console.WriteLine (fs.Position);
// 20000
}
In this example, the underlying stream advances 20,000 bytes after reading just 1 byte, thanks to the read-ahead buffering. We could call ReadByte another 19,999 times before the FileStream would be hit again.
Coupling a BufferedStream to a FileStream , as in this example, is of limited value because FileStream already has built-in buffering. Its only use might be in enlarging the buffer on an already constructed FileStream .
Closing a BufferedStream automatically closes the underlying backing store stream.
Stream Adapters
A Stream deals only in bytes; to read or write data types such as strings, integers, or XML elements, you must plug in an adapter.
Text Adapters
TextReader and
TextWriter are the abstract base classes for adapters that deal exclusively with characters and strings.
using (FileStream fs = File.Create ("test.txt"))
using (TextWriter writer = new StreamWriter (fs))
{
writer.WriteLine ("Line1");
writer.WriteLine ("Line2");
}
using (FileStream fs = File.OpenRead ("test.txt"))
using (TextReader reader = new StreamReader (fs))
{
Console.WriteLine (reader.ReadLine()); // Line1
Console.WriteLine (reader.ReadLine()); // Line2
}
Because text adapters are so often coupled with files, the File class provides the static methods CreateText , AppendText , and OpenText to
shortcut the process:
using (TextWriter writer = File.CreateText ("test.txt"))
{
writer.WriteLine ("Line1");
writer.WriteLine ("Line2");
}
using (TextWriter writer = File.AppendText ("test.txt"))
writer.WriteLine ("Line3");
using (TextReader reader = File.OpenText ("test.txt"))
while (reader.Peek() > −1)
Console.WriteLine (reader.ReadLine());
This also illustrates how to test for the end of a file (viz.
reader.Peek() ). Another option is to read until reader.ReadLine returns null.
You can also read and write other types such as integers, but because TextWriter invokes ToString on your type, you must parse a string when reading it back:
using (TextWriter w = File.CreateText ("data.txt"))
{
w.WriteLine (123);
// Writes "123"
w.WriteLine (true);
// Writes the word "true"
}
using (TextReader r = File.OpenText ("data.txt"))
{
int myInt = int.Parse (r.ReadLine());
// myInt == 123
bool yes = bool.Parse (r.ReadLine());
// yes == true
}
Character encodings
TextReader and TextWriter are by themselves just abstract classes with no connection to a stream or backing store. The
StreamReader and
StreamWriter types, however, are connected to an underlying byte-oriented stream, so they must convert between characters and bytes. They do so through an Encoding class from the System.Text namespace, which you choose when constructing the StreamReader or StreamWriter . If you choose none, the
default UTF-8 encoding is used.
StringReader and StringWriter
The StringReader and StringWriter adapters don’t wrap a stream at all; instead, they use a string or StringBuilder as the underlying data source. This means no byte translation is required—in fact, the classes do nothing you couldn’t easily achieve with a string or StringBuilder coupled with an index variable. Their advantage, though, is that they share a base class with StreamReader / StreamWriter . For instance, suppose we have a string containing XML and want to parse it with an XmlReader .
The XmlReader.Create method accepts one of the following:
- A URI
- A Stream
- A TextReader
So, how do we XML-parse our string? Because StringReader is a subclass of TextReader , we’re in luck. We can instantiate and pass in a StringReader as follows:
XmlReader r = XmlReader.Create (new StringReader (myString));
Binary Adapters
BinaryReader and
BinaryWriter read and write native data types: bool , byte , char ,
decimal , float , double , short , int , long , sbyte , ushort , uint , and ulong , as well as
string s and arrays of the primitive data types.
BinaryReader can also read into byte arrays. The following reads the entire contents
of a seekable stream:
byte[] data = new BinaryReader (s).ReadBytes ((int) s.Length);
This is more convenient than reading directly from a stream, because it doesn't require a loop to ensure that all data has been read.
Compression Streams
Two general-purpose compression streams are provided in the System.IO.Compression namespace: DeflateStream and GZipStream.
DeflateStream and GZipStream are decorators; they compress or decompress data from another stream that you supply in construction. In the following example, we compress and decompress a series of bytes, using a FileStream as the backing store:
using (Stream s = File.OpenRead ("compressed.bin"))
using (Stream ds = new DeflateStream (s, CompressionMode.Decompress))
for (byte i = 0; i < 100; i++)
Console.WriteLine (ds.ReadByte());
// Writes 0 to 99
Even with the smaller of the two algorithms, the compressed file is 241 bytes long: more than double the original! Compression works poorly with “dense,” nonrepetitive binary filesdata!
In the next example, we compress and decompress a text stream composed of 1,000 words chosen randomly from a small sentence. This also demonstrates chaining a backing store stream, a decorator
stream, and an adapter (as depicted at the start of the chapter in Figure 15-1), and the use asynchronous methods:
string[] words = "The quick brown fox jumps over the lazy dog".Split();
Random rand = new Random();
using (Stream s = File.Create ("compressed.bin"))
using (Stream ds = new DeflateStream (s, CompressionMode.Compress))
using (TextWriter w = new StreamWriter (ds))
for (int i = 0; i < 1000; i++)
await w.WriteAsync (words [rand.Next (words.Length)] + " ");
Console.WriteLine (new FileInfo ("compressed.bin").Length);
// 1073
using (Stream s = File.OpenRead ("compressed.bin"))
using (Stream ds = new DeflateStream (s, CompressionMode.Decompress))
using (TextReader r = new StreamReader (ds))
Console.Write (await r.ReadToEndAsync()); // Output;
In this case, DeflateStream compresses efficiently to 1,073 bytes—slightly more than 1 byte per word.
Compressing in memory
Sometimes you need to compress entirely in memory. Here’s how to use a
Memory Stream for this purpose:
byte[] data = new byte[1000];
// We can expect a good compression
// ratio from an empty array!
var ms = new MemoryStream();
using (Stream ds = new DeflateStream (ms, CompressionMode.Compress))
ds.Write (data, 0, data.Length);
byte[] compressed = ms.ToArray();
Console.WriteLine (compressed.Length);
// 113
// Decompress back to the data array:
ms = new MemoryStream (compressed);
using (Stream ds = new DeflateStream (ms, CompressionMode.Decompress))
for (int i = 0; i < 1000; i += ds.Read (data, i, 1000 - i));
The using statement around the DeflateStream closes it in a textbook fashion, flushing any unwritten buffers in the process. This also closes the MemoryStream it wraps —meaning
we must then call ToArray to extract its data.
Working with Zip Files
new feature in Framework 4.5 -
ZipArchive and
ZipFile classes
ZipFile is a static helper class for ZipArchive;
ZipFile ’s
CreateFromDirectory method adds all the files in a specified directory into a zip file:
ZipFile.CreateFromDirectory (@"d:\MyFolder", @"d:\compressed.zip");
whereas
ExtractToDirectory does the opposite and extracts a zip file to a directory:
ZipFile.ExtractToDirectory (@"d:\compressed.zip", @"d:\MyFolder");
File and Directory Operations
FileInfo offers an easier way to change a file’s read-only flag:
new FileInfo (@"c:\temp\test.txt").IsReadOnly = false;
Here are all the members of the
FileAttribute enum that
GetAttributes returns:
Archive, Compressed, Device, Directory, Encrypted, Hidden, Normal, NotContentIndexed, Offline, ReadOnly, ReparsePoint, SparseFile, System, Temporary
File security
The
GetAccessControl and
SetAccessControl methods allow you to query and change the operating system permissions assigned to users and roles via a FileSecurity object (namespace System.Security.AccessControl ). You can also pass a
FileSecurity object to a
FileStream ’s constructor to specify permissions when creating a new file.
In this example, we list a file’s existing permissions, and then assign execution permission to the “Users” group:
FileSecurity sec = File.GetAccessControl (@"d:\test.txt");
AuthorizationRuleCollection rules = sec.GetAccessRules (true, true,
typeof (NTAccount));
foreach (FileSystemAccessRule rule in rules)
{
Console.WriteLine (rule.AccessControlType); // Allow or Deny
Console.WriteLine (rule.FileSystemRights); // e.g., FullControl
Console.WriteLine (rule.IdentityReference.Value); // e.g., MyDomain/Joe
}
var sid = new SecurityIdentifier (WellKnownSidType.BuiltinUsersSid, null);
string usersAccount = sid.Translate (typeof (NTAccount)).ToString();
FileSystemAccessRule newRule = new FileSystemAccessRule
(usersAccount, FileSystemRights.ExecuteFile, AccessControlType.Allow);
sec.AddAccessRule (newRule);
File.SetAccessControl (@"d:\test.txt", sec);
The Directory Class
he static Directory class provides a set of methods analogous to those in the File class—for checking whether a directory exists (
Exists ), moving a directory (
Move ), deleting a directory (
Delete ), getting/setting times of creation or last access, and getting/setting security permissions. Furthermore, Directory exposes the following static methods:
string GetCurrentDirectory ();
void
SetCurrentDirectory (string path);
DirectoryInfo CreateDirectory (string path);
DirectoryInfo GetParent
(string path);
string
GetDirectoryRoot (string path);
string[] GetLogicalDrives();
// The following methods all return full paths:
string[] GetFiles
(string path);
string[] GetDirectories
(string path);
string[] GetFileSystemEntries (string path);
IEnumerable<string> EnumerateFiles (string path);
IEnumerable<string> EnumerateDirectories (string path);
IEnumerable<string> EnumerateFileSystemEntries (string path);
FileInfo and DirectoryInfo
The static methods on
File and
Directory are convenient for executing a single file or directory operation. If you need to call a series of methods in a row, the FileInfo and DirectoryInfo classes provide an object model that makes the job easier.
FileInfo offers most of the File ’s static methods in
instance form—with some additional properties such as Extension , Length , IsReadOnly , and Directory —for returning a DirectoryInfo object. For example:
FileInfo fi = new FileInfo (@"c:\temp\FileInfo.txt");
Console.WriteLine (fi.Exists); // false
using (TextWriter w = fi.CreateText())
w.Write ("Some text");
Console.WriteLine (fi.Exists); // false (still)
fi.Refresh();
Console.WriteLine (fi.Exists); // true
(fi.Name); // FileInfo.txt
(fi.FullName); //c:\temp\FileInfo.txt
(fi.DirectoryName); //c:\temp
(fi.Directory.Name); //temp
(fi.Extension); // .txt
(fi.Length); // 9
fi.Encrypt();
fi.Attributes ^= FileAttributes.Hidden; //(toggle hidden flag)
fi.IsReadOnly = true;
Console.WriteLine (fi.Attributes); // ReadOnly,Archive,Hidden,Encrypted
Console.WriteLine (fi.CreationTime);
fi.MoveTo (@"c:\temp\FileInfoX.txt");
DirectoryInfo di = fi.Directory;
Console.WriteLine (di.Name); // temp
Console.WriteLine (di.FullName); // c:\temp
Console.WriteLine (di.Parent.FullName); // c:\
di.CreateSubdirectory ("SubFolder");
Here’s how to use DirectoryInfo to enumerate files and subdirectories:
DirectoryInfo di = new DirectoryInfo (@"e:\photos");
foreach (FileInfo fi in di.GetFiles ("*.jpg"))
Console.WriteLine (fi.Name);
foreach (DirectoryInfo subDir in di.GetDirectories())
Console.WriteLine (subDir.FullName);
