Community Server

Review of Concurrent Programming on Windows

bobcalco — Sat, 03 Jan 2009 14:25:00 GMT

I am in the middle of Joe Duffy's Concurrent Programming on Windows, but already I know I'm going to be blogging a lot about this book and its sublime subject matter here for some time.

First impression: It's the best book about the internals of multi-threaded processing ever written. It's the best book in this space because it's both practical, hands-on with its examples, and fairly systematic in its coverage---without becoming dreadfully dull reading. Basically, he illustrates his ideas with real, and often very useful and lively code, rather than terse formulas or a barrage of academic points-of-interest.

I learned at least one thing I didn't know --- namely, that despite supposedly heroic efforts Microsoft has not yet been able to expose fibers to .NET developers. This is appalling, actually, but Duffy covers them anyway, with a great example of how to write a user-mode scheduler leveraging fibers in C++. Kudos to him for that!

More than anything, he is thorough. If you don't know what a semaphore is, or how monitors work, or why liveness and correctness are always at risk---if, architecturally speaking, you don't clearly understand the trade offs between various concurrency strategies---you almost certainly will if you read this book. There is no longer an excuse for any developer, even one untrained in computer science, not to be able to wrap his or her head around multi-threaded programming. Like Prego--it's in this book, and its quite digestible. (Yes, that was faint praise for Prego---but a sincere expression of enthusiasm for the book!)

One thing I plan to do in this blog is take the concepts for a ride on different platforms, using different programming languages. I am currently considering a plan-of-attack, including a general-purpose, onoing example that will help bring the issues into focus.

Recurring Events in SQL

anachostic — Fri, 12 Dec 2008 23:02:48 GMT

I recently had a little piece of SQL code that I thought I would post and it reminded me of something I did a long time ago (5 yrs) and thought I'd write something on it. Anyway, first here's the little piece without much celebration.

This SQL code answers the question "What date is the third Wednesday in November 2008?" This code could be wrapped in a function pretty easily.

declare @month int,@year int,@weekday int,@weekOffset int

set @month=11 -- (Month of date to check) 
set @year=2008 -- (Year of date to check)
set @weekday=5 -- 1-7 (Sun-Sat)
set @weekOffset=1 -- 1-5 (Week of month)

declare @currentMonth datetime
declare @firstDay int
declare @newDate datetime

-- get the first day of the month
set @currentMonth=cast(cast(@month as char)+'/1/'+cast(@year as char) as datetime)
set @firstDay=datepart(dw,@currentmonth)

-- Add the number of days from the first day
set @newDate=dateadd(dd,7*(@weekOffset-1)+
    case 
        when @weekday>=@firstDay then @weekday-@firstDay
        when @weekday<@firstDay then 7+@weekday-@firstDay
    end, @currentmonth)

select @newdate

The bigger idea I had was for recurring events in SQL, like for a calendar. Assume the following structure of an event table

CREATE TABLE dbo.Events(
    ID int IDENTITY(1,1) NOT NULL ,
    StartDate datetime,
    EndDate datetime,
    Title varchar(255),
    Description varchar(2000),
    RepeatValue char(1),
    RepeatInterval int
)

So you have a bunch of events defined that start and end on a range of dates. The RepeatValue would be: D,W,M,Y for Day, Week, Month, Year. The RepeatInterval would be the number of Days/Weeks/Months between repeats. Add in some sample data

insert events select '1/1/08','12/31/08','Each Month All Year','','M',1
insert events select '1/1/08','6/30/08','Bi-weekly Half Year','','W',2
insert events select '7/1/08','12/31/08','Weekly Half Year','','W',1
insert events select '2/1/08','3/1/08','Every other Day in Feb','','D',2
insert events select '7/4/08','12/31/2999','July 4th','','Y',1

This is the stored proc I came up with to return the events for a given month:

CREATE Procedure [dbo].[sp_ListRecurringCalendarEvents]
@viewdate datetime
as

set nocount on

declare @checkdate datetime

create table #tempCalendar(
    id int,
    startdate datetime,
    enddate datetime,
    title varchar(255),
    recurrance bit default 0
)

-- Insert Standard events
insert #tempCalendar
select e.id,e.startdate,e.enddate,e.title,0
from events e
where datepart(mm,e.startdate)=datepart(mm,@viewdate) 
    and datepart(yy,e.startdate)=datepart(yy,@viewdate)
    and repeatinterval=0

-- Insert all recurrances
set @checkdate=cast(cast(datepart(mm,@viewdate) as char) 
    + '/1/' 
    + cast(datepart(yy,@viewdate) as char) as datetime)

-- Loop through each day in month and see if it is a recurrance
while datepart(mm,@viewdate)=datepart(mm,@checkdate)
    begin
    insert #tempCalendar
    select e.id,
    case when RepeatValue='D' then dateadd(dd,datediff(dd,startdate,@checkdate),startdate)
     when RepeatValue='W' then dateadd(ww,datediff(ww,startdate,@checkdate),startdate)
     when RepeatValue='M' then dateadd(mm,datediff(mm,startdate,@checkdate),startdate)
     when RepeatValue='Y' then dateadd(yy,datediff(yy,startdate,@checkdate),startdate)
    end,
    e.enddate,e.title,
    case when @checkdate=e.startdate then 0 else 1 end
    from events e
    where e.RepeatInterval<>0 
        and
        -- Daily (Check to see if the number of day interval matches
        case when RepeatValue='D' then (cast(datediff(dd,startdate,@checkdate) as decimal)
            /cast(RepeatInterval as decimal)) else 1 end =
            case when RepeatValue='D' then (datediff(dd,startdate,@checkdate)/RepeatInterval) else 1 end
        and 
        -- Weekly (check to see if the week interval matches and the weekday is the same)
        case when RepeatValue='W' then (cast(datediff(ww,startdate,@checkdate) as decimal)
            /cast(RepeatInterval as decimal)) else 1 end =
            case when RepeatValue='W' then (datediff(ww,startdate,@checkdate)/RepeatInterval) else 1 end
            and case when RepeatValue='W' then datepart(dw,startdate) else 1 end    =
            case when RepeatValue='W' then datepart(dw,@checkdate) else 1 end
        and 
        -- Monthly (check to see that the month interval is the same and the day of the month is the same)
        case when RepeatValue='M' then (cast(datediff(mm,startdate,@checkdate) as decimal)
            /cast(RepeatInterval as decimal)) else 1 end =
            case when RepeatValue='M' then (datediff(mm,startdate,@checkdate)/RepeatInterval) else 1 end
            and case when RepeatValue='M' then datepart(dd,startdate) else 1 end    =
            case when RepeatValue='M' then datepart(dd,@checkdate) else 1 end
        and 
        -- Yearly (Check to see if the day of the year is the same)
        case when RepeatValue='Y' then datepart(dy,startdate) else 1 end =
            case when RepeatValue='Y' then datepart(dy,@checkdate) else 1 end
        and 
        e.startdate <= case when RepeatValue='D' then dateadd(dd,datediff(dd,startdate,@checkdate),startdate)
            when RepeatValue='W' then dateadd(ww,datediff(ww,startdate,@checkdate),startdate)
            when RepeatValue='M' then dateadd(mm,datediff(mm,startdate,@checkdate),startdate)
            when RepeatValue='Y' then dateadd(yy,datediff(yy,startdate,@checkdate),startdate)
            end 
        and 
        e.enddate >= case when RepeatValue='D' then dateadd(dd,datediff(dd,startdate,@checkdate),startdate)
            when RepeatValue='W' then dateadd(ww,datediff(ww,startdate,@checkdate),startdate)
            when RepeatValue='M' then dateadd(mm,datediff(mm,startdate,@checkdate),startdate)
            when RepeatValue='Y' then dateadd(yy,datediff(yy,startdate,@checkdate),startdate)
            end

    -- Increment the date
    set @checkdate=dateadd(dd,1,@checkdate)

    end    

-- return the results
select * from #tempCalendar

-- Clean up
drop table #tempCalendar

The proc is called passing a date within the month to be displayed. The results include the ID of the event (for linking), the start date of the event, the end date of the event (not so relevant), the event title, and whether this is a recurring event or not (in case you want to display it differently or disable editing of events that aren't the original).

So, many years later, I still find it intriguing that I was doing stuff like that.

Your Code Is Very Important To Us, Please Continue To Wait...

anachostic — Wed, 26 Nov 2008 00:41:52 GMT

It was asked of me once: Why isn't my "please wait" form doing anything? The form had an animation, but the animation never moved. The main form code would show the "please wait" form, run some code, then hide the "please wait" form. As it turned out, the "please wait" form was being blocked by a database call and some other stuff in the main form. There was never a chance for a DoEvents or redraw or anything.

The quick answer is, you have to show the other form in another thread, so it can operate freely and redraw itself. The longer answer is, you have to deal with threading.

Threading is very easy in VB.NET, but some people want answers, not lessons, so I ended up writing this quick class to show a form in another thread. It's pretty simple. You instantiate the class by passing in an instance of your "please wait" form , then call show(), do your work, then call hide(). Like so:

Dim f as New BackgroundForm(new frmWait)

f.Show()
' Do some stuff
f.Hide()

Only one caveat: you can't have the variable for the BackgroundForm class at the class level. It must be declared, instantiated, and used in the same method. Otherwise, you will get Cross-Threading errors.

So, here's the class you can use for yourself, if you prefer answers to lessons.

Public Class BackgroundForm
    Dim displayForm As Form
    Dim t As System.Threading.Thread
    Dim isActive As Boolean
    Shared lockObject as New Object

    Public Sub New(ByVal f As Form)
        displayForm = f
    End Sub

    Public Sub Show()
        t = New System.Threading.Thread(AddressOf ShowForm)
        t.IsBackground = True
        t.Start()
        isActive = True

    End Sub

    Public Sub Hide()
        isActive = False
    End Sub

    Private Sub ShowForm()
        displayForm.Show()

        Do While isActive AndAlso displayForm.Visible
            System.Threading.Thread.Sleep(100)
            SyncLock lockObject
            Application.DoEvents()
            End SyncLock
        Loop

    End Sub

End Class

I made a slight change to the code to protect against thread stomping. If you have an animated GIF in your "please wait" form, and you have a more than one "please wait" form active at once, there is the chance you will get a "object is in use elsewhere" error. This happens when the image for the current form is being changed - in the DoEvents - but another form is being closed/disposed, taking the same image with it.

By wrapping the DoEvents in a SyncLock that is shared among all instances, this is prevented. I couldn't get the error to happen unless I opened over 20 windows and had them closing at scheduled times, but your luck may be worse.

Autosize. No, not the control, the text.

anachostic — Sun, 16 Nov 2008 16:37:31 GMT

It's nice that .NET controls have an auto-size property so you don't have to worry about overflow and all. But what about cases where you have a fixed layout? Well, that's simple, you turn autosize off and fix the control to the size you need.

That's half the story. What about the text that's inside it? Now you know I'm talking to marketing people when I say that there are times you want the text to be as big as possible within that control. But you can't just set the font to a huge size, because sometimes you'll have more text to display and the font size must sadly be reduced.

To accommodate this, I made a quick method that brute-forces the correct font size in the control. basically, stepping down the size of the font until it fits. I know loops like this are cheap, poor programming, and I did give consideration to doing some hard math to calculate the proper font size based on the initial size, but sometimes not getting hung up on performance can be liberating.

    Private Sub ResizeText(ByVal c As Control)
        Dim currentSize As Size
        Dim currentFont As Font

        currentFont = c.Font

        Do
            currentSize = TextRenderer.MeasureText(c.Text, currentFont, _
                c.Size, TextFormatFlags.WordBreak)

            If currentSize.Width > (c.Width - c.Margin.Horizontal) _
                OrElse currentSize.Height > (c.Height - c.Margin.Vertical) Then

                currentFont = New Font(currentFont.FontFamily, _
                    CSng(currentFont.Size - 0.5), currentFont.Style, currentFont.Unit)
            Else
                Exit Do

            End If

        Loop While currentFont.Size >= 1

        c.Font = currentFont

    End Sub

Casting Upwards

anachostic — Wed, 05 Nov 2008 00:52:40 GMT

When binding business objects to a datagrid, often you have a need to display some information that is not directly exposed by the object itself. Maybe it's a calculated value, maybe it's something nested deeper in the object. When faced with this issue, there are a few different action paths you can take. You can add extra read-only properties to your business object to support the extra view information. You can create a new class that inherits from the class you are displaying and put the extra properties in there. Or you can handle the CellFormatting event in the datagrid and change the displayed values manually. One of the downsides of using a new derived class with extra properties is that you can't cast a base class to it. You could cast down to the base class, but no casting up.

Here is a technique that is closest to the second option listed above and side-steps the upcasting problem. I dislike the first option because it clutters the business object with UI-specific code. Going with option 2 is only slightly better, while you can populate the correct display-specific object and return it from your business logic layer, either you have to have a method that return the derived type, or you will have to cast it to its correct type in the UI. Even then, your business layer still contains UI logic.

So, keeping things separated, the derived display-specific class should be defined in the UI layer. This means it will have extra read-only properties for use with databinding. The business layer will return the basic object(s), so it will be up to us to convert these to UI-friendly versions. There are two problems with converting the object: not all the object state may be exposed via public properties, and those properties may contain logic. It would be best to copy the object by its internal state - private variables.

On first thought, working with the private variables means the code must be inside the source object and the destination object. This would be tedious to do, passing in the destination object, then sending the source object's private variables to the destination so the destination object can manipulate its own private variables. Yuck. However, using Reflection, the job gets a whole lot easier.

Here's a small class with a method to convert one class to another by copying its private and public fields. The properties are intentionally excluded since they may contain logic that modifies the internal state. You should use this technique with care and know exactly what it does and does not do. Basically, it copies values from one instance of a class to another. This is fine for simple classes, but it's not going to resolve references for you.

Consider ClassA with a private field of type ClassB. ClassB maintains a private variable with a reference to ClassA, so that it can manipulate all of its "parent's" state and logic. If you use this technique to cast ClassA to ClassAA, because you want an extra property to display some info from ClassB, you're in for some fun results if you change some data in ClassAA. This is because ClassB still has a reference to ClassA, not ClassAA.

Public Class UpCaster
    Shared Sub CastUp(ByVal sourceObj As Object, ByVal destinationObj As Object)
        Dim values As New Dictionary(Of String, Object)
        Dim props() As Reflection.FieldInfo 

        props = sourceObj.GetType.GetFields(Reflection.BindingFlags.NonPublic _
            Or Reflection.BindingFlags.Static _
            Or Reflection.BindingFlags.Instance _
            Or Reflection.BindingFlags.Public) 

        For Each p As Reflection.FieldInfo In props
            values.Add(p.Name, p.GetValue(sourceObj))
        Next 

        props = destinationObj.GetType.GetFields(Reflection.BindingFlags.NonPublic _
            Or Reflection.BindingFlags.Static _
            Or Reflection.BindingFlags.Instance _
            Or Reflection.BindingFlags.Public) 

        For Each p As Reflection.FieldInfo In props
            If values.ContainsKey(p.Name) Then p.SetValue(destinationObj, values(p.Name))
        Next 

    End Sub 

End Class

Zune XNA Example 2

anachostic — Mon, 08 Sep 2008 00:55:57 GMT

The last post was really long and I think I tried to do too much in one post. So, here's something smaller.

In the last example, we created a custom class for a stack of cards. One thing I want to point out about that class is that it had a shuffle method. Said another way, the class was in charge of shuffling the cards inside it. The XNA game that used the card stack didn't contain the logic for shuffling the cards. That's one of the keys of creating objects - they take care of themselves. To reinforce that concept, here's a game that has no playability. Here's the game description:

The game will display a dot on the screen. The dot will be a random color and random size, and will appear in a random location.

Objects? I only read "dot". "Screen" is part of the game, so we already have that object built in. Properties? Dot seems to have a color, size and location. The term "random" refers to the value of those properties, so we don't need to concern ourselves with using that as a property. We will need to remember it when we write the code though. Methods? None that are mentioned. But the way we are going to write this, we are going to have the dot draw itself, without any involvement from the game. So the dot knows where it is, what size it is, and what color it is. It seems perfectly capable of drawing itself.

Into the code. This is our RandomDot class:

using System;
using System.Collections.Generic;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Graphics;
using Microsoft.Xna.Framework.Content;

namespace DrawRandomDots
{
    class RandomDot:DrawableGameComponent
    {
        Random _rndm;
        Game _parent;
        SpriteBatch _spriteBatch;
        List<Color> _availableColors;
        Texture2D _dot;
                
        public RandomDot(Game game):base(game)
        {
            _rndm = new Random();
            _parent = game;
            _availableColors=new List<Color>();
        }

        protected override void LoadContent()
        {
            _spriteBatch = new SpriteBatch(_parent.GraphicsDevice);
            _dot=_parent.Content.Load<Texture2D>("Dot");
            _availableColors.AddRange(new Color[]{Color.Blue,Color.DarkBlue,Color.DarkGreen,
                Color.DarkRed,Color.Green,Color.LightBlue,Color.Magenta,Color.Navy,
                Color.Orange,Color.Purple,Color.Red,Color.White,Color.Yellow});
              
            base.LoadContent();
        }

        public override void Draw(GameTime gameTime)
        {
            Rectangle size;
            Color color;
                        
            size = new Rectangle(_rndm.Next(0, 240), _rndm.Next(0, 320), _rndm.Next(1, 10), _rndm.Next(1, 10));
            color = _availableColors[_rndm.Next(0, _availableColors.Count)];

            _spriteBatch.Begin();
            _spriteBatch.Draw(_dot, size, color);
            _spriteBatch.End();
            
            base.Draw(gameTime);
        }
    }
}

It's not a lot of code. Here's the line-by-line:

Line 1-5 (using...): These are namespaces we are going to be using in our class. Because we put them in using statements, we don't need to type all that out when we include the type name. Notice I brought in some XNA namespaces, which is something we didn't need in the CardStack class. Because this class is actually going to be drawing, we will be using classes in these namespaces.

Line 7 (namespace...): This is the namespace for our custom class. Like I mentioned before, this is the name of the project I'm working in. If you create a project under a different name, you need to update this to match.

Line 9 (class...): This is the beginning of our class definition. It's called RandomDot. The colon indicates we are inheriting from DrawableGameComponent, so we're going to get a lot of code for free. DrawableGameComponent is part of the Microsoft.XNA.Framework namespace, which is included in the using statements above.

Line 11 (random...): This is the variable for the random number generator we'll use for the color, location, and size. Our game description specifically says random.

Line 12 (Game...): This variable will hold the game that the dot is being included in. I used the name "parent" to emphasize a point. The dot class cannot exist on its own. It is dependant on a game. Without a game to draw the dot in, the dot class cannot be drawn. Similar to how a person cannot be created without a parent, the dot class cannot be created without a game.

Line 13 (spritebatch...): This variable will hold the spritebatch that we draw to. Because this class is going to handle drawing itself, we need a spritebatch to draw to.

Line 14 (List...): This variable will hold the colors that the dot can be. We will select one color at random from this list.

Line 15 (Texture2D...): This variable will hold the graphic that we will draw to the spritebatch.

Line 17 (public...): This is our constructor for the class. Unlike the cardstack class, this class has only one constructor and it requires that a game object be passed into it. By doing this, we enforce the rule that this class cannot exist without a game. The colon and "base(game)" code show that the base class, which is DrawableGameComponent, needs to receive the game object as well. If you do not include this on the constructor, you'll get a "...does not contain a constructor that takes '0' arguments" error.

Line 19 (_rndm...): We instantiate our random generator. If we don't, we get "object reference not set" errors. This is the most common error you will probably experience, so it's always important to ensure your objects get instantiated.

Line 20 (_parent...): We are going to hold on to a reference to the parent game, because other methods in our class will need it. So we store it in a variable.

Line 21 (_availablecolors...): We instantiate our list so we can start adding colors into it.

Line 24 (public...): This is a method that is provided by DrawableGameComponent, but we want to add additional code into it. To do that, we use the keyword "override". That tells .NET: "when using this class, run this code instead of the usual code." In Visual Studio it's easy to do overrides, you type in "override" and Intellisense gives you a list of methods you can override. Pick one and the proper code is written for you.

Line 26 (_spritebatch...): We're in the LoadContent method, which only gets run once. So we are going to create the objects that we need to draw the dot. We need a spritebatch to draw to. We can get this from the game that the dot belongs to, if we had passed it in from the game, or we can create a brand new one from the GraphicsDevice of the game. We're going to to the latter, but there are drawbacks to doing it this way. For such a simple game, it's not a concern. Notice we use the _parent variable to reference the game that the dot belongs to.

Line 27 (_dot...): Here we load the image that will be our dot. Again, we are using the functionality the game provides us. This is a single pixel image called dot.png.

Line 28-30 (_availablecolors...): Here we are assigning the colors to the list. This is a shorthand way of putting in a bunch of items into a list. The code "new Color[]{... ... ...}" creates an array of colors. That array isn't assigned to any variable. That code is within the statement ".AddRange()", so the array is sent into the AddRange method which adds all the array items into the list. This is another case where you need to read the line from inside out.

Line 32 (base...): In line 24, I said that we are replacing the usual code with our own using "override". But DrawableGameComponent has a lot of functionality in it for this method already. We don't want to have to rewrite it all; we want to use it. In this line we are calling the LoadContent method in DrawableGameComponent, so we're not losing anything. So what we've done it not so much replace the functionality of LoadContent, but more enhance it with additional code.

Line 35 (public...): Here's another case where we want to inject additional code into the generic DrawableGameComponent. So we override the Draw method also. The Draw method has a parameter for the current game time. When we override it, we have to include that parameter also. When we override, the parameters must be just like the base class.

Line 37 (Rectangle...): We need a variable to hold the size and location of the dot. The rectangle class will do both for us together. If we just wanted location, we could use Vector2

Line 38 (Color...): This variable will hold the color of the dot. Sometimes in the code, you'll see some shorthand versions used and others we'll assign the value to a variable when we use it. If we're going to assign it to a variable, it's usually because we need to use that value more than once, or by using an extra variable, we add clarity to the code. In this case, we are doing the latter.

line 40 (size=...): Here we create the rectangle to hold the location and size of the dot. Notice the X and Y coordinates are the width and height of the screen and the width and height of the rectangle are 1 through 10. Each value is randomly selected.

Line 41 (color=...): Here we pick a color at random from the list of available colors. Notice the upper limit of the random.Next statement is the count of the list. This means we can add or remove colors from the list without having to change this line. If we hard-coded 10 in there (because we put 10 colors in the list), we'd have to change it if we altered the size of the available colors list. Is this hypocritical that the width and height used in the screen size in line 40 are hard-coded values yet I'm saying hard-coded values are bad? Absolutely. You can get the current screen size from XNA, and maybe we'll do that later.

Line 43 (spritebatch...): Before writing to a spritebatch, we need to call Begin on it.

Line 44 (spritebatch...): Here we draw the dot texture at the specific location in the specific color. Notice in line 38 how I mentioned clarity? Imagine replacing the variables with all the code from line 40 and 41. The extra variables are worth it. Also, notice we are specifying what color to draw the dot? But our dot is an image and it has a color already, right? Yes, but if your image is just white, the color parameter tints the image in the color you specify. So you can have a solid color image and draw it in any color as long as the original texture is white. So you don't need a green dot, a red dot, a blue dot. You just make a white dot and draw it in the color you want.

Line 45 (spritebatch...): When we are done drawing to a spritebatch, we need to call End.

Line 47 (base...): Like line 32, we want the regular code for the Draw method of DrawableGameComponent to run as well, so we call that here. You don't have to always have this line at the end. You may want the base code to run first or somewhere in the middle of your additional code. In these cases, we run it last.

So now you have a class that draws itself. You'll see that in the game code, there is actually very little we have to write now. Here's the code for the game:

using System;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Content;
using Microsoft.Xna.Framework.Graphics;
using Microsoft.Xna.Framework.Input;
using Microsoft.Xna.Framework.Storage;

namespace DrawRandomDots
{
    public class Game1 : Microsoft.Xna.Framework.Game
    {
        GraphicsDeviceManager graphics;
        SpriteBatch spriteBatch;

        RandomDot dot;

        public Game1()
        {
            graphics = new GraphicsDeviceManager(this);
            Content.RootDirectory = "Content";
            dot = new RandomDot(this);
            this.Components.Add(dot);
        }

        protected override void LoadContent()
        {
            spriteBatch = new SpriteBatch(GraphicsDevice);
        }

        protected override void Update(GameTime gameTime)
        {
            if (GamePad.GetState(PlayerIndex.One).Buttons.Back == ButtonState.Pressed)
                this.Exit();

            base.Update(gameTime);
        }

        protected override void Draw(GameTime gameTime)
        {
            graphics.GraphicsDevice.Clear(Color.Black);

            base.Draw(gameTime);
        }
    }
}

Once again, I'll only discuss the additions I made to the Zune game template. I added a dot.png to the Content folder of the project. The image is just a single white pixel.

Line 15 (RandomDot...): Here is the variable for our custom class. We're only going to have one for right now.

Line 21 (dot=...): We need to instantiate our class so we can use it. We pass in the current game in the constructor by using "this".

Line 22 (this...): This is something new. We are going to add the random dot as a component of the game. This means that it will get drawn automatically for us. If you think of any normal application, like Microsoft Word, can you imagine how hard it would be to write the main program if you had to maintain all the little pieces like menus, toolbars, rulers, status bars, popup messages? Instead, these individual pieces are built independently and they manage themselves. That's what you have done. You have built a component that manages itself.

That's it. Only three changes. Well, I changed the background color to black in line 40, too. But the point is, because the RandomDot class handles drawing itself, the main program does not need to. As you make games that have lots of different parts, you will be crushed under the weight of the code in the Draw method is you try to do it all in the main game class. So when you're writing a card game, the cards should draw themselves. But wait, how will the cards know where to be drawn? If there's five cards, how will they know about each other so they can choose a location that doesn't overlap with each other? We'll do that in a later post, but the quick answer is: another class that has a list of cards inside it. That class knows about the five cards. The cards don't know and will never know about each other.

A Zune Game Sample in C#

anachostic — Sat, 06 Sep 2008 21:45:00 GMT

I will start out with a disclaimer: I am not a C# programmer; I am a VB programmer. I have heard countless people say, "Why don't you just learn C# and be done with it. They're just the same." After this little experiment, I'm a little more convinced, but I am certainly more productive in VB, so that's where I will stay.

So what is the purpose of this post (and likely future posts)? Because I have a need to share knowledge and help others, and there is a need in the Zune community to get people up to speed with XNA programming. As previous posts mention, I'm doing Zune programming in VB. I realize this is not for everyone, least of all beginners. So I will apply some concepts that are universal to .NET programming in a C# format in order to promote some skill building.

The format I am going to use is to put out all the code first, then walk through it line-by-line in excruciating detail. This way, you can hopefully skip over lines that you understand and just jump to a concept that is unfamiliar. So scan through the full code listing, mentally compiling it and visualizing different parameters being passed and what the code path would be. If you aren't sure what the effect of a line would be, check the detail for that line.

Ok. The first sample I could think of, because of elevated interest in it, is cards. This sample mixes a bit of basic programming with a sample of "what's the best way to...". The sample is pretty basic, but can be improved upon rather easily because it's going to be built with planned expansion.

First some general terms and some design discussion. The term "class" refers to a code structure. A class should define something of interest to your program. When you talk about your program to other people, the nouns you use in the description are excellent candidates for classes. Consider the statement, "My game has a car driving around a course dodging potholes and barricades". Thinking in code, you should immediately think: "I need classes for car, course, pothole, and barricade."

The term "property" refers to a piece of information contained in the class. This is data that makes the instance of the class unique. These are the adjectives in your description. The statement: "The player's car can be red or blue and can be upgraded with weapons." You should be visualizing that the car class needs at least two properties: color and weapons. When visualizing properties, don't consider the values of the properties - like red and blue - think of how to store the values.

Finally, "method" refers to an action that the class can perform, either on itself or on another instance of a class. When describing games, these are the verbs. "The car can speed up, turn left and right, and brake." You should be thinking "accelerate, turn, brake" for methods on the car class.

Now, that you can translate descriptions into classes, you are now officially a software architect. Welcome to the world of software design. Here's the description of the "game" we are going to create:

This game has a deck of cards. You will draw a card and the value will be displayed to you. After all the cards have been drawn, the deck is reshuffled and the first card is drawn again. That's it.

When I read that, I think the objects should be: card and card deck. The properties of card should be: value. The methods of card deck should be: shuffle. What about having "display" on the card class? In this case, we are going to assign all display duties to the game class. And XNA has a method for that: Draw(). That doesn't mean you will always do it that way. Maybe in a future revision I'll illustrate the difference.

Hopefully you interpreted the description the same as I did, because that's how we're going to build it. In this first example, the card class is going to be handled by String. So all of our cards are going to be strings. Later we'll make it a class of its own. The class we are going to focus on is the card deck.

When you think of a deck of cards, you can draw parallels to lots of programming concepts: arrays, lists, collections, but the most appropriate programming concept is the Stack. A stack is an auto-sizing array where you can only put things on and take things off. You can only get at the last thing put on the stack, just like you can only take off the last card put on a deck of cards. It's a perfect fit. .NET gives us a Stack class that we can inherit from and put in our extra method: shuffle. Here's the code:

using System;
using System.Collections.Generic;

namespace CardsSample
{
    class CardStack<T> : Stack<T>
    {
        Random rndm;

        public CardStack()
        {
            rndm = new Random();
        }

        public CardStack(CardStack<T> cards)
        {
            rndm = new Random();

            while (cards.Count > 0)
            {
                this.Push(cards.Pop());
            }
        }

        public void Shuffle()
        {
            List<T> cards = new List<T>();
            T currentCard;
            int randomIndex;

            while (this.Count > 0)
            {
                cards.Add(this.Pop());
            }

            while (cards.Count > 0)
            {
                randomIndex = rndm.Next(0, cards.Count);
                currentCard = cards[randomIndex];
                cards.RemoveAt(randomIndex);

                this.Push(currentCard);
            }

        }
    }
}

Here's the explanation:

Line 1&2 (using ...): We have using statements to reduce the amount of code we have to type. If we didn't have them, every time we defined a Random variable, we would have to use System.Random, and instead of typing Stack<T>, it would be System.Collections.Generic.Stack<T>.

Line 4 (namespace...): Classes have namespaces to help organize the code library. If we didn't have a unique namespace and we just happened to call our class "Stack", how would .NET know whether any variable of type "Stack" was ours or the one in System.Collections.Generic? Yes, that's right. The using statements are namespaces.

Line 6 (class...): This is the beginning of the definition of our class. There are a couple of elements on this line that are not going to be something you use all the time. The first is <T>. This makes our class a Generic class that can be used for any data type. Big-picture, we want this to be a stack of cards, but right now, we are using strings as our cards. This class could have been written using <string> instead of <T>, but when we converted it to work with our later card class, we'd have to change it all again. My making it generic, we can use strings, cards, or anything else. The other element is the colon and the class following it. This indicates that our class is inheriting from Stack<T>. We get a lot of pre-built functionality from Stack, and the <T> designation means that however we define our class (strings, cards, whatever), the Stack class we are sitting on top of will be the same.

line 8 (Random...): This is the definition of our random number generator we'll use when shuffling. This variable is only seen and used inside the class, and that's how we want it. No other class should care about the deck's random number generator.

Line 10 (public...): This is our constructor for the class. When another piece of code calls "New CardStack<string>()", this is the code that runs. The term "constructor" means the method that is called when creating a new instance of a class.

Line 12 (rndm =): This is where we instantiate the random class. If we didn't do that, the shuffle method would error with an "Object reference not set" error. Just by declaring a variable doesn't always mean it's ready to use.

Line 15 (public ...): This is another constructor, but this one takes a parameter. If you've ever created a new class and Visual Studio shows the yellow help with (1 of 15), showing all the different ways you can create a new object, that's what we're doing here. If you write code new "CardStack<string>(", you will be prompted for either of the two ways to create it, either empty, by passing no parameters, or filled, by passing in another CardStack instance. Why would we want to create a filled CardStack? That will be shown in the game code.

Line 17 (rndm...): Just like Line 12

Line 19 (while...): The constructor is there to fill the stack with items from another stack. The while loop will do this. Notice we are looking at "cards.count". What is "cards" and how do we know we can use it? This is the parameter we passed in: its name is cards and it's type is CardStack<T>. "cards" is only usable within this method. If we edit line 15 and change "cards" to some other name, we also need to change it on this line so it is referenced properly.

Line 21 (this.push...): This looks like magic to a first-timer. The key is to read it from the inside out. Inside the parentheses, the statement "cards.Pop()" will return the top item off the card stack that was passed in. You might think you need to assign it to a variable first, but you don't. So right now, we have a card floating in limbo. The statement outside that, "this.Push()", captures that card and puts it on the top of the stack for the current class instance. The keyword "this" always means the current instance of the class. It makes sense that you would need that when you have 10 instances of the CardStack class going at once.

Line 25 (public void...): This is our shuffle method. Because we used the keyword "public", other code can see it and execute it. Because we used "void", other code knows that nothing is coming back when this method gets called. The method definition doesn't specify any parameters.

line 27 (List<T>...): This variable is going to temporarily hold all the items in the stack. Remember, we can only access the last item on a stack, so that's going to seriously limit how random we can be. A list however, can have any of its items accessed at any time. So that's what we'll use. The <T> designation means to use the exact same type that was used when we defined the cardstack class - string, card, whatever. We are also creating a new instance of the list on the same line. Since we are going to use it right away, we can do this.

Line 28 (T ...): This might look a little odd because there is no data type T. But this is actually a reference to the <T> we've been using all over the place. Whatever we use when we create CardStack<T> gets used here. CardStack<string> defines currentCard as string, CardStack<card> would define currentCard as card.

Line 29 (int...): This is used to hold a random number when we pull from the temporary list. We can't do magic like on line 21, because we need this number on two different lines.

Line 31 (while...): We're going to loop through all the items in the current stack and put them in the temporary list. We'll keep pulling items out until its empty. There are lots of other clever ways to do this and later code will probably use them.

Line 32 (cards.add...): This is just like line 21, except instead of adding to a stack, we're adding to a list. It's nice that a lot of classes use the same method names.

line 36 (while...): Instead of looping through the internal collection, we're going to loop through the temporary collection. We're going to keep looping through it until its contents have been emptied back into the stack.

line 38 (randomindex...): We get a random number somewhere between 0 and the number of items in the temporary list. Every time we loop, that count is going to drop by one, so we need to evaluate it each time. That's why we don't get the count of temporary items before we start looping. So it will be 0 through 10, then 0 through 9, and so on.

Line 39 (currentcard=...): This is where we get a random card (or string) from the temp list based on the random number we got.

line 40 (cards.removeat...): We want to remove the selected item from the temporary list so we don't get it again. Don't worry about losing it, we still have a reference of it in currentCard. But we need to do something with currentCard, because that's the last place we can find that card.

Line 42 (this.push...): And now we've secured the card in the current CardStack. We've put the card on the top of the stack, so if we did a .Pop() right now, we would pull it back off.

So if the stack had 1,2,3,4,5 in it, a pop would pull off 5. To shuffle, we popped off all the items and put them in a list. The list looks like 5,4,3,2,1 because we pop from the stack backwards. Then depending on the numbers from the random generator, the stack may have any order of values because we pulled from the list in a random order, never exceeding the length of the list.

Wow, that was exhausting. But we're not done yet. Now that we understand the cards are going to be strings, and the deck of cards is based on a Stack class, we can build the game. Writing an XNA game is different from writing a Windows application. Windows apps do nothing until something is done to them. That's event-driven programming. XNA games are running all the time, constantly checking for input and drawing the screen. It takes a different mindset to design a game. I'm certainly not going to say I'm great at game writing, because I tend to shoehorn in event-driven concepts into the process. Regardless, let's see the game code. A lot of this is boilerplate from the XNA template, so I'll only highlight the lines that I added:

using System;
using System.Collections.Generic;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Audio;
using Microsoft.Xna.Framework.Content;
using Microsoft.Xna.Framework.Graphics;
using Microsoft.Xna.Framework.Input;
using Microsoft.Xna.Framework.Media;
using Microsoft.Xna.Framework.Storage;

namespace CardsSample
{
    public class Game1 : Microsoft.Xna.Framework.Game
    {
        GraphicsDeviceManager graphics;
        SpriteBatch spriteBatch;

        CardStack<string> drawCards;
        CardStack<string> discardCards;
        string currentCard;
        SpriteFont font;
        bool isButtonPressed;

        public Game1()
        {
            graphics = new GraphicsDeviceManager(this);
            Content.RootDirectory = "Content";

            drawCards = new CardStack<string>();
            discardCards = new CardStack<string>();
        }

        protected override void Initialize()
        {
            base.Initialize();
        }

        protected override void LoadContent()
        {
            spriteBatch = new SpriteBatch(GraphicsDevice);

            font = Content.Load<SpriteFont>("MainFont");

            for (int i = 0; i < 10; i++)
            {
                drawCards.Push("Card " + i.ToString());
            }

            drawCards.Shuffle();
        }

        protected override void Update(GameTime gameTime)
        {           
            if (GamePad.GetState(PlayerIndex.One).Buttons.Back == ButtonState.Pressed)
                this.Exit();

            if ((!isButtonPressed) && (GamePad.GetState(PlayerIndex.One).Buttons.A == ButtonState.Pressed))
            {
                if ((drawCards.Count == 0) && (discardCards.Count > 0))
                {
                    drawCards = new CardStack<string>(discardCards);
                    drawCards.Shuffle();
                }

                currentCard = drawCards.Pop();
                discardCards.Push(currentCard);
                
                isButtonPressed = true;
            }
            
            if (GamePad.GetState(PlayerIndex.One).Buttons.A != ButtonState.Pressed)
                isButtonPressed = false;

            base.Update(gameTime);
        }

        protected override void Draw(GameTime gameTime)
        {
            string msg;

            msg = "Your card is {0}." + Environment.NewLine   
                + "There are {1} cards remaining" + Environment.NewLine  
                + "and {2} cards have been drawn.";

            graphics.GraphicsDevice.Clear(Color.Black);
            spriteBatch.Begin();
           
            if (currentCard != null)
            {
                spriteBatch.DrawString(font, string.Format(msg, currentCard, drawCards.Count, discardCards.Count), 
                     new Vector2(20, 100), Color.White);
            }else
            {
                spriteBatch.DrawString(font, "Press Center to draw card.", 
                       new Vector2(20, 100), Color.White);
            }
            
            spriteBatch.End();

            base.Draw(gameTime);
        }
    }
}

Line 18 (CardStack...): We get to use our custom class. This is going to hold the cards that we pull from. Notice we actually specify a real type - string - in the definition. When we convert to using a real class for our cards, we only need to change the definition here. We don't need to touch the CardStack class at all.

Line 19 (cardStack...): Another of our classes. This is going to hold the cards after we pull from the draw stack. Think of two piles of cards (two stacks of cards, ahem). That's what we have in these two variables.

line 20 (string...): This is going to be the variable for our currently-drawn card. We pop from the draw stack and put it into current card and push it onto the discard stack.

Line 21 (spritefont...): We need a font to display our text. To add a font, right-click the Content item in Visual Studio's Solution Explorer and Add New item, choose SpriteFont and give it the name MainFont, then edit the FontName field in the SprinteFont file to say Arial, and the Size to 12.

Line 22 (bool...): This variable is used to remember when the draw button is pressed. Remember, XNA is always checking the input and the way our code is, we only want to draw a card when the button is initially pressed, not every time XNA checks to see the button is pressed.

line 29/30 (drawcards...): We instantiate the variables that will hold our cards. Like the random variable in our CardStack, if we don't, we'll get "Object Reference Not Set" errors.

Line 42 (font=...): We load the font from the Content manager. The name we pass in is the same as the filename we saved the spritefont as, "MainFont". Notice we indicate what type of content we are loading in the <SpriteFont> designation. When we get to having graphics in the game, we'll use different values.

Line 44 (for...): This is a loop that will put cards into the draw stack. The parameters of the For loop basically are: use an int and start it at zero, loop while the value is less than 10, and increment it by one each time. If you wanted number 10 through 20, you'd change the first parameter to initialize the int to 10 and change the second parameter to check for less than 20. If you only wanted even numbers, change the last parameter to increment by 2 (i+=2).

line 46 (drawcards...): Here we push the new card onto the draw stack. remember, we're just using text right now. when we use a card class, we'll do a bit more here. The card uses the value of the loop variable to uniquely identify it.

line 49 (drawcards...): We get to use our shuffle method. During the loop we were putting on cards 0 through 10, so our draw stack would look like 1,2,3,3,4,5,6,7,8,9,10. The first card we pulled off the deck would be 10, then 9, and anything but random. So we shuffle it first. Don't trust it? Shuffle it a few more times.

Line 57 (if ((button...): We're in the update method. This gets called very frequently and is used to check for input. This particular line checks two things: is the draw button already pushed? Or more accurately, has it been held down? If it is, don't bother drawing a card. If the button was not held down previously and the button is now pushed down, then let's draw a card.

line 59 (if ((drawcards...): We know we are going to draw a card, but what if there are no cards to draw - they are all discarded. Then we reinitialize the draw stack and reshuffle. Now you see why we created a constructor that accepts a CardStack parameter. Sure, we could have looped through the discard stack, popped everything off, pushed it onto the draw stack, then shuffled it, but what if you needed to do that somewhere else, like if we had a reset button? Then you'd have to write the same code again there. Let the stack class handle it.

Line 61 (drawcards...): We create a new instance of the CardStack class and pass it the discard stack to build from. I hear you veterans mumbling about wasting memory by creating another instance. No one cares at this point in a tutorial.

line 62(drawcards...): We shuffle the rebuilt draw stack before we draw our first card.

Line 65(currentcard...): Whether we are dealing with a fresh deck or not, we always pull a card off and store it in the currentcard variable. We need it stored because another method is going to display its value.

Line 66(discardCards...): So we don't lose the card, we put it on the discard stack. we still have our reference to the card in currentCard.

Line 68 (isbuttonpressed...): We want to remember that the card has already been drawn for this button push, no matter how long the button is held in for. Line 57 makes sure we don't keep drawing cards when we set this variable.

Line 71 (if (gamepad...): If we don't reset the isButtonpressed variable, we'll never get a second press. So we check to see if the button has been lifted, and if so, reset the variable. Then line 57 will catch the next button press.

line 72 (isbuttonpressed...): reset the button check variable to false. The update method gets called like 60 times a second, so there should be no noticeable delay between button presses.

line 79 (string...): We are going to use a format string to display the message. It's a little cleaner code. This will be what we store the string in.

line 81-83 (msg=...): Here we are defining the format string. We use {0}, {1}, and {2} are placeholders for where we want to insert text.

Line 86 (spritebatch...): We have to do Begin and end when writing to a spritebatch.

line 88 (if...): For presentation, we want to check of the current card is nothing. This only happens the first time you start the program, because you haven't drawn any cards yet. So this checks whether the current card is nothing so the right message is displayed. This will still work when we create a card class.

line 90 (spritebatch...): We draw the string to the spritebatch for display. This uses the string.format function, which replaces the placeholders in our format string with the value specified: current card, count of draw stack, and count of discard stack. Notice we didn't write any code for counting the size of the stacks. It came free when we inherited from Stack. Our .DrawString method is using the spritefont we loaded back on line 42. We specify a position to write the text using a Vector2 class, and specify a color.

line 91 (else...): If the first card isn't drawn yet...

Line 93 (spritebatch...): Draw the instruction to draw the first card. This time we are using a regular string instead of the formatted string.

Line 96 (spritebatch...): You have to do End when done writing to a spritebatch. If you forget you'll get a specific error message telling you that you forgot.

That's it. You're done. And I'm exhausted. I'm not sure I'll be able to keep up this level of detail with future posts, and maybe no one needs it, but you can't be sure. This post explained some very basic programming topics and tried to provide as much "why" as possible. It also gave an example of inheritance and generics, two things not really meant for first-timers, so if you got it, great. That was part of the "best way to do it" instruction.

The next step is to create the card class and use it instead. That's actually pretty easy, so maybe it will include some other tips.

During proofreading, I noticed something that might really cause trouble for copy/pasters. When you create a new C# project, your default namespace is created from the name of the project. Notice in my code samples, the namespace is CardsSample. If you create a project with a name that is different from CardsSample, you will get errors when building. Why?

As I mentioned earlier, namespaces are used to organize code. So the cardStack class can exist in many different places in your project as long as each definition is in its own namespace, so they won't conflict. If you would copy and paste the code into your project, you now effectively have two namespaces in your single project. There's nothing wrong with that, but that's not what you want or expect. To fix it, change the namespace in the code you are pasting to match the name of your project. Or, you can look at the namespace being used in the file Program.cs and copy that to your pasted code.

Too Many Items In Combo Box: When One Is Just One Too Many

anachostic — Wed, 03 Sep 2008 01:46:36 GMT

I got to troubleshoot a dumb error message today. The error was "Too many items in combo box." The situation was anything but. I was only adding one item.

So I got it working and I wanted to find out why it happened in the first place. The error it should have returned was "Value cannot be NULL" because that was the root of the problem. So here's a distilled piece of code to illustrate the problem. Create a form and put a combo box on it. The code for the form should look like:

Public Class Form1

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) _ 
        Handles MyBase.Load

        Dim d As New DisplayItem
        ComboBox1.Items.Add(d)

    End Sub

End Class

Public Class DisplayItem
    Public Name As String

    Public Overrides Function ToString() As String
        Return Name
    End Function

End Class

The problem is the combo box is trying to display DisplayItem.Name, because that is what the ToString says to do, but the value of Name is Nothing. You can fix this by setting the value of Name to String.Empty or something else. The odd thing is that you can also fix the problem by commenting out the ToString override. To figure this out, I fired up Reflector and went to see what was going on behind the scenes.

This particular situation is basically bypassing all the safe value checks done when adding an item to a list control. I suppose Microsoft should add a test case for this scenario, but really, if the programmer is attentive, this shouldn't happen. I, naturally, happen to be inattentive.

Behind the scenes of the Add method of the Items collection, the first check is in the AddInternal method. Since we're passing in an instance of DisplayItem, it passes that check. The next step is in the NativeAdd method. At this point, we've done our NULL checks and it is assumed we can convert the object to a string. This method now calls GetItemText.

GetItemText parses the properties of the object passed in and gets the string value. If the DisplayMember property is not set, the control uses the ToString value of the object itself. Because we overrode ToString, the control trusts us and returns the value from ToString, the Name value. This turns out to be Nothing - Oops! We’ve already passed the check for Nothing, so this sends bad data to the Win32 API, bubbling a failure error code back to NativeAdd. If NativeAdd gets anything but a success, it always returns the message “Too Many Items In Combo Box”. But the real reason is that you snuck a Nothing past the initial validation.

Interestingly, if the DisplayMember is set, and the value of the property is Nothing, it is handled properly in GetItemText. If we converted Name to a private field and made a public property, then set the DisplayMember of ComboBox1, it would work. If your display member is another object that overrides the ToString function, you can get around that check as well and return Nothing, causing a failure.

The simple solution for this error message is to avoid NULL values. The bottom line is to have .ToString always return a string, never Nothing.

IsInRole While Disconnected; No Longer IsInHole

anachostic — Wed, 27 Aug 2008 00:46:26 GMT

In an application I am continuing to write, during startup, the user's security is determined by the user's security groups and permissions are granted within the program based on the group membership. This has worked fine. Then one day in an airport I wanted to work on some documentation and I quickly discovered that I could not run the program. I could not run the program because I was not on the VPN and the Active Directory (AD) groups could not be enumerated to check my permissions. At the time, I was kind of grateful I couldn't do any work and didn't give it much more thought. Our application is not designed to be run in a disconnected fashion. You have to be on the VPN to get to the database server and use it. Lately, I revisited the problem and decided to resolve it.

In my particular instance, I had a database running locally, but I was missing an Active Directory server to read from. I had my cached credentials, shouldn't that be enough? Well, it is, if you do it the hard way.

Everyone should know that a user or a group is just a name. There is an ID behind that group or user, which allows you to rename the group/user without breaking anything. Behind the scenes, Windows always uses the ID. The name is just for your benefit.

Using the excellent tool WhoAmI, I saw the following (don't chide me for running as Administrator):

C:\>whoami /groups /sid

[Group  1] = "700CB\Domain Users"  S-1-5-21-2454202000-1896829455-2950045386-513
[Group  2] = "Everyone"  S-1-1-0
[Group  3] = "DA3\Debugger Users"  S-1-5-21-854245398-1547161642-725345543-1007
[Group  4] = "DA3\Offer Remote Assistance Helpers"  S-1-5-21-854245398-1547161642-725345543-1004
[Group  5] = "BUILTIN\Users"  S-1-5-32-545
[Group  6] = "BUILTIN\Administrators"  S-1-5-32-544
[Group  7] = "NT AUTHORITY\INTERACTIVE"  S-1-5-4
[Group  8] = "NT AUTHORITY\Authenticated Users"  S-1-5-11
[Group  9] = "LOCAL"  S-1-2-0
[Group 10] = "700CB\Domain Admins"  S-1-5-21-2454202000-1896829455-2950045386-512

When disconnected form the LAN (you might need to reboot while disconnected to clear the cache), it looks like:

C:\>whoami /groups /sid

[Group  1] = ""  S-1-5-21-2454202000-1896829455-2950045386-513
[Group  2] = "Everyone"  S-1-1-0
[Group  3] = "DA3\Debugger Users"  S-1-5-21-854245398-1547161642-725345543-1007
[Group  4] = "DA3\Offer Remote Assistance Helpers"  S-1-5-21-854245398-1547161642-725345543-1004
[Group  5] = "BUILTIN\Users"  S-1-5-32-545
[Group  6] = "BUILTIN\Administrators"  S-1-5-32-544
[Group  7] = "NT AUTHORITY\INTERACTIVE"  S-1-5-4
[Group  8] = "NT AUTHORITY\Authenticated Users"  S-1-5-11
[Group  9] = "LOCAL"  S-1-2-0
[Group 10] = ""  S-1-5-21-2454202000-1896829455-2950045386-512

My local group names were resolved to their names, but my domain group names couldn't resolve because AD was unreachable. My cached profile still had the SIDs though.

So, if I could do an IsInRole check using the SID instead of the domain group name, I'd be golden. And this is just what I did.

Imports System.Security.Principal
Imports System.Threading

Dim sid As New SecurityIdentifier("S-1-5-21-1859785585-1835888107-1082013118-1025")
Dim p As WindowsPrincipal = CType(Thread.CurrentPrincipal, WindowsPrincipal)

MsgBox(p.IsInRole(sid))

So I took the SID for the groups I was testing for and tested for them instead. Obviously, if an admin deleted and recreated the group thinking nobody would notice, it would be hell to troubleshoot, so if you're not fully in control of your environment, you might want to steer clear. Maybe do the SID after checking IsNetworkAvailable to reduce the exposure to failure?

But for me, it works like a champ, and now I can work in airports. Hmmm. Why did I figure this out again?

Columns Autosize: Listview in List View

anachostic — Sat, 23 Aug 2008 02:04:56 GMT

Here's another little snip of code I couldn't find online when I needed it. Geez, when I became a programmer, I wasn't thinking I'd have to actually figure things out on my own. That's a lot of work.

Anyway, the problem I faced was when I had a Listview control on a form and I changed views from anything to List view, the columns of the items were really small, so I'd get ellipses' after all the entries. Sure, I could just set the column with to some obnoxious amount like 500, but that's a waste of space. So after searching and seeing a bunch of postings about using a Win32 API call to autoresize the column, then getting disappointed because it was for VB 4/5/6, I just hacked through it.

The Details view has an autosize feature, but apparently they didn't extend it to the List view. But we can still make use of it. Why not switch to detail, set the autosize of column 0 to true, then measure how wide it makes the column, then use that as the column width in List view? That's a dumb idea. Who would do something like that. Oh, what do you know, it works.

Dim maxSize As Integer

lstItems.View = View.Details
lstItems.Columns(0).AutoResize(ColumnHeaderAutoResizeStyle.ColumnContent)
maxSize = lstItems.Columns(0).Width

lstItems.View = View.List
lstItems.Columns(0).Width = maxSize
lstItems.Refresh()

Yay.

Questions; Twenty of 'em

anachostic — Tue, 12 Aug 2008 01:41:02 GMT

I was thinking of that interesting little pocket game, 20Q, where the game knows so many words and is able to figure out what word you're thinking of in 20 questions. For the most part, it's not too bad. It must really be pretty complex to be able to figure out all those words.

I've brainstormed on this idea before and whenever I start the code for it, I lose interest, but it doesn't stop me from kicking it around in my head over and over.

So pretty much, you have a database table full of answer words, which also has a numeric PK, you have a table of questions, with numeric PK, and you have a table with a matrix of every question and every answer PK and a response value (1=yes, 0=no). That would be the basic data structure. For data population, you could fill the answer table with random nouns pulled from any dictionary file. That's simple. For the questions table, you'd have to enter as many random questions as you could think of. Random. And LOTS. Don't think of the answer, just think of any question. Then after you've fried your mind with questions, do a Cartesian join to insert the PKs into the matrix table, leaving the response field null. Now the fun part. Make a form with big Yes and No buttons. The form will read each row in the matrix table, joined with the question and answer table and you respond yes or no to each question/answer combination. This should take a few days.

After sleeping off that headache and letting your eyes reset, it would be time to test coverage. Do all of your questions provide a unique combination for every answer? Does that get the wheels in your head spinning as to how to create such a uniqueness test? I'm going to post my untested first idea, because nothing spurs responses better than posting something wrong. My idea would be have an outer query (this is in .NET) for each answer PK, then an inner query that reads the question PK for each response that is yes (sorted by question PK, filtered by answer PK), loop through the inner query and concatenate the question PKs together into a big string and store it (maybe in a hashtable) with the answer ID as the value. If you hit a dupe, then you have two answers that have the exact same yes responses.

After you've tested for full coverage and removed any answers that can't be exclusively answered by a series of responses, the goal is to present the most efficient question first. What is the most efficient question? The one that removes the most answers from the potential answer pool. This is another fun mental exercise. At the simplest level, it's the count of answers grouped by question filtered by all previous questions that don't match previous responses. Something like (in pseudo-code)

select Questions.ID,count(Answers.ID) AnswerCount
from QAMatrix
join Answers on QAMatrix.AnswersID=Answers.ID
join Questions on QAMatrix.QuestionsID=Questions.ID
where Questions.ID=[previous question id] and QAMatrix.Response<>[previous response value]
and Questions.ID=[previous question id] and QAMatrix.Response<>[previous response value]
...
group by Questions.ID
order by 2 desc

I think that would do it. The first question in the results should have the highest answers being trimmed out. Looking at it now during a proofread, I suppose it might need enhanced so that it identifies the balance of yes/no responses in that count. It won't do much good if 90% the answers were yes. Well... depending on the answer, that's a potential risk with a potential huge reward - wiping out 90% of the possible answers. Would an advanced 20Q program take more risks early or later or would it play middle-of-the-road and always shoot for a 50% split? Design details for the interested. Another 2nd version detail might be to remove any questions that don't split the answers well enough. Maybe that's how 20Q ended up with such weird questions like "Does it grow hair?"

And that's about all the fun stuff I've been thinking about and not doing lately.

Why does Vista SP1 show me I have 4 GB of memory available?

tom.fuller — Wed, 23 Jul 2008 21:01:00 GMT

This is one of those deep nerdy windows internals posts that will interest some and make other say "wow, Tom needs to spend more time away from his computer". There is actually a good reason why I've dug into this topic and it stems from my recent hardware allocation in my new job. When I started I received a new Lenovo laptop with 4 GB of memory and my base image came with Vista 32 bit. I didn't think too much of it and loaded everything up. This was about 6 months ago so there was no Vista SP 1 at the time.

To my dismay when I opened my computer properties I saw that I only had 3070 MB of available physical memory. So somehow I lost an entire gig of RAM. I spoke with some of my other colleagues that were also new hires and they didn't have the same issue. Many of them had HP or Dell notebooks and they had more like 3.5 GB of available physical memory on their machines.

It was right around this time that I got my first deep internals class for my new job. In this class we learned all of the tips and tricks for doing advanced perfmon analysis of memory leaks, disk performance issues, CPU bottlenecks, the list goes on and on. Well the instructor was just incredible so when there was a break in the class I asked him about this issue. He very quickly explained that the reason you see different amounts of available ram was because of the chipset and reserved space for device drivers (video cards typically the biggest hogs).

Ok, no big deal then right. Lesson learned, when you look to purchase a laptop that can support up to 4 GB of physical memory make sure you find out how much it will make available to windows. Then, Vista SP1 came out and I couldn't help but notice when I went to the computer properties the window now said I had 4 GB of ram available. Well isn't that interesting, someone must have lied to me!!!! Actually it turns out that with SP1 we decided to show the installed RAM instead of the available RAM. This is very confusing as it's never been that way for any previous OS version. If you want to learn some more about it you can take a look at this kb article http://support.microsoft.com/kb/946003/.

Now my story here is one of confusion and ignorance. If you're the type that wants to really understand this stuff then you have to go the best windows internals guy on the planet. Of course I'm talking about Mark Russinovich, he wrote a long blog post just this week (which is what reminded me I wanted to blog about this) that gives a complete explanation of this issue along with some other really cool stuff about RAM limits on other OS versions. He also shows a screenshot from the largest test server in the world used to define the limits for memory in 64 bit windows (currently 2 TB because there was nothing any higher available that we could test with). This is a must read http://blogs.technet.com/markrussinovich/archive/2008/07/21/3092070.aspx. If you just want the technical detail behind the 4GB issue I talked about above then scroll down to the "32-bit Client Effective Memory Limits" section.

So the main lessons here are:

1) If you are still planning to run 32-bit windows clients and you plan to install more than 3 GB of RAM then not all hardware is created equal. Make sure you ask how much physical RAM is made available to windows. This becomes even more important when you start doing a lot of Virtual Lab work as I often do. It sure would be nice to get that third Win 2K3 server running with a full GB of RAM. Why didn't I install the 64 bit version of Vista Ultimate again?

2) The 64-bit client version of windows is really the only way to ensure you can start to have enough RAM for doing a lot of virtualization work. It's also the only way to get in and see how many gaps your hardware has left in the device driver ranges (see Mark's review of this). 64-bit is the future ... embrace it ... love it ... live it.

Blog consolidation complete.

tom.fuller — Wed, 23 Jul 2008 20:01:00 GMT

There are two reasons why I went ahead and deleted my other blogs today. The first is that I finally came to grips with the reality that managing three blogs is not very realistic. That was probably evident by the fact that each of those blogs had about 2 postings. So I'll be adding some new blog posts in my good old SOA focused blog that have maybe some out of bounds topics (like support, debuggin, and SQL BI).

The second reason for consolidating my other blogs is that I'm finding that the technology gods, in their infinite wisdom, have carved out a path for me in my new role at Microsoft that leads to SOA. Three areas I've been spending a lot of time on the last two months are WCF, BizTalk and the Managed Services Engine (codeplex.com/servicesengine). You may have noticed this in the form of some more technical postings on WCF and BizTalk recently. Well that's only the beginning. I have quite a few more active support cases where I'm learning really interesting stuff and I'm about to head out to Seattle for 10 days of training. Half of that training will be on all of our SOA related bits from the SOA Solutions team. Needless to say I'm really stoked.

Expect to see a lot of posts coming from me in the near future about all this stuff I'm working on. I know I've said this before but I'm now focused and very excited because of the technology area I'm beginning to venture into. Mr. SOAPitStop is back baby!!!!

Gnarly issue setting up BizTalk 2006 R2 in a 2-Tier configuration on Virtual Server 2005

tom.fuller — Wed, 09 Jul 2008 18:08:00 GMT

Before I forget to do this I wanted to post about this issue because it cost me a 1/2 day of debugging and I hope to save you this agony. It actually turns out there's a nasty little issue when you use Virtual Server 2005 differencing disks to quickly stand up lab enviorments. In my new role with Microsoft support I'm constantly setting up 1, 2, 3, and sometimes 4 node lab environments so differencing disks are pretty much a must. Of course if you want to do this and still need the ability to join these servers to a test domain, then you have to make sure you rename / resid them. That's where I've always relied on newsid from the sysinternals tools.

I've always wondered if this was a hack (versus a full sysprep) and I wondered when it would jump up and bite me. I noticed in IIS that the anonymous user account does not get recreated to use the new computer name but that's never impacted me (I sort of figured a local account really isn't going to make much of a diff). Well, I have now finally found an issue that is ugly. It turns out that if you are going to build an app server / db server both from the same base OS image you are going to have some issues with MSDTC. I ran into this as I started to run through the BizTalk configuration tools. I kept getting the following error:

2008-07-08 17:44:58:0896 [ERR] WMI Failed in pAdmInst->Create() in CWMIInstProv::PutInstance(). HR=c0c025b3
2008-07-08 17:44:58:0906 [ERR] WMI WMI error description is generated: Exception of type 'System.EnterpriseServices.TransactionProxyException' was thrown.
2008-07-08 17:44:58:0936 [INFO] WMI CWMIInstProv::PutInstance() finished. HR=c0c025b3
[5:44:58 PM Error BtsCfg] d:\depot2300\mercury\private\mozart\source\setup\btscfg\btswmi.cpp(358): FAILED hr = c0c025b3

[5:44:58 PM Error BtsCfg] Exception of type 'System.EnterpriseServices.TransactionProxyException' was thrown.

At least this error helped me narrow down the root cause of the problem which is obviously System.EnterpriseServices. So I started looking at all my settings for MSDTC and nothing seemed out of the norm. All the right security settings and network dtc access settings were there. So it was time to do a quick search with live.com (that's right ... I said live.com ... google is so 5 years ago :)

It turns out the problem is with the underlying CID settings for MSDTC. It looks like newsid doesn't go in and take care of these. Fortunately I wasn't the first one to come across this because I don't know if I would have come up with these steps. A big bravo goes out to Wade Wegner who wrote a great overview of the problem and the steps to fix it. You can read the full description and steps to identify wether or not this is your issue here: http://blog.wadewegner.com/2007/08/13/quotWarningTheCIDValuesForBothTestMachinesAreTheSamequot.aspx.

If you just want the solution you can follow these quick steps on both machines. (taken from Wade's blog).

Use Add Windows Components, and remove Network DTC.

Go to the command line and run: MSDTC -uninstall
Go to the registry and delete the MSDTC keys in HKLM/Software/Microsoft/Software/MSDTC, HKLM/System/CurrentControlSet/Services/MSDTC, and HKEY_CLASSES_ROOT\CID. *
Reboot
Go to the command line and run: MSDTC -install
Use Add Windows Components, and add Network DTC.
Go to the command line and run: net start msdtc **

* for me the CID settings were deleted when I did the -uninstall so you shouldn't need to remvoe the last ones. Also, this does mean you should delete the key at the top level and purge all the subkeys, that freaked me out a little at first but of course we're just talking about a test lab here so I figured why not try it. It worked like a champ.

** This step wasn't required for me as the service was already running following me adding the network DTC access from the windows components installer.

Yet Another Ambitious Project (YAAP)

bobcalco — Sun, 29 Jun 2008 11:17:00 GMT

Lately I have been working in Erlang (forgive me, Tom), and in particular on a distributed web-based system that exposes much of its functionality over a RESTful interface. The web server has been Yaws and the database Mnesia, both of which are also implemented in Erlang, and are quite complex and interesting in their own right. Using Erlang's OTP (Open Telecom Platform) framework it's relatively easy to create and configure complex, robust client-server/state-machine applications, taking advantage of Erlang's native support for massive concurrency, high-availability and fault tolerance. Anyway, building such a system gives you a sense of where the grunt work is, and grunt work is a great candidate for code generation to mitigate the problem of time-to-market. This time-to-market problem is an issue that is felt quite poignantly when you are working in a startup on borrowed time, as I am at the moment. :) So currently I am devoting a little R&D time each day to the development of a code generation tool for RESTful applications in Erlang, which I call "shenkuo." The reason for the name is two fold: 1.) I plan to release it as open source via the Erlware project (www.erlware.org), and 2.) they like funky and exotic names based on real people that have some (obscure or obvious) relevance to the functionality of the project. The reason for Erlang is that it rocks for any problem involving concurrency and network communications--I mean really ROCKS. (More on that subject will be forthcoming in my Functional Programmer blog.) Shen Kuo was an ancient Chinese polymath who, among other things, came up with the concept of "due north" (a great aid to the compass, by the way), described the first ideas behind movable type, and invented several branches of what we in the west today call the social sciences. Whatever do these wacky and diverse things have to do with what I'm building? Well, RESTful applications are all the rage in social networking sites, and they are used to do all kinds of various and sundry things in real world "Web 2.0" architectures. REST has attained something of the aura of "due North" as a philosophy behind web application design. So the name "shenkuo" for a code generation framework for RESTful applications based on Erlang/OTP seems as good as any. And it just so happens that my first such distributed application is in the sphere of web publishing, so the reference to movable type in Shen Kuo's curriculum vitae seems to close the deal for me. I will be posting more about this YAAP in this space as I work on it. One of its key requirements will be to generate client proxy libraries to consume the generated RESTful interfaces in addition to the interfaces themselves, and I'll be focusing in this space on .NET clients (particularly WPF and Silverlight), for those of you here wondering why Tom tolerates my presence on his Microsoft SOA blog site. :)