Coding, is? Fun!

The Two-Hop issue and Web apps

2010-04-03T00:23:00.000-07:00

Consider these two scenarios:
1. Your ASP.NET web app has an <identity impersonate="true"/> in the web.config. You are accessing SQL server from your web app using Windows authentication. Your users' windows accounts DO have access provided in SQL server. IIS is set to Integrated Windows authentication.
Your application works locally or in a Dev server as long as SQL server is installed in the web server itself. You move the app to User Acceptance environment, where SQL server is in a different box from the web server. Permissions are unchanged. Yet, your web application throws an authentication error when accessing SQL server - it is a classic case of "it works in dev, but not in other environments".

2. Your ASP.NET web app accesses a network file share for some files. The web app, again, has <identity impersonate="true/> in the web.config. Your users have permission to the network file share. IIS is set to Integrated Windows authentication.
Your app works fine in dev, as long as your browser is in the same box as the web server. Thus it works well in development. You move the app to a different server, SRV 1. All of a sudden, even for you, when you browse the app from your computer and hit SRV 1, you get an authentication error - suddenly, your network share is not accessible.
But it works if you open the browser in the SRV 1 box itself!

When encountering the above two and various similar scenarios, I have seen developers think that the problem is specific to SQL server, their web app, IIS or the network share. But what you are probably facing is the dreaded Two-Hop issue - it is an important infrastructural issue. It has nothing to do with specific services like SQL Server or IIS.

The Two-Hop issue
Very specifically, what you are attempting in the above two scenarios is this: you are trying to traverse, using an user's windows identity, from your browser to the web server (First Hop) and then from the web server to the SQL server (or network share or LDAP server - this is the Second Hop).
The user (you) have entered your windows network user name and password in your box, and you have been authenticated. With this authentication (that is, with your password) you can ONLY go from your box to one other (any other) server that you have permission to. Once your id reaches the web server, the web app is trying to pass it on to the SQL server or network server(that is the meaning of <identity impersonate="true"/>). BUT IT CANNOT DO SO! Why not? Because in a windows network, the primary authentication token (in other words, your password) is only valid for one server to another. The recipient server cannot pass it on to another server and so forth.
This is a very important security restriction - and it has nothing to do with specific services like IIS or SQL server. It is the nature of the windows security infrastructure.
When IIS or SQL server "authenticate" you, and they use Integrated Windows Authentication, then the authentication process is handled by the underlying operating system. The operating system treats it simply as a request from one server to another. The web app's request to SQL server is no different from opening a network share on the SQL server machine from the web server - both these requests use the SAME authentication mechanism.
Since the flow is from server to server, the Two-Hop issue kicks in. Now let us see the above scenarios again:
Your browser makes the request to web app - you enter your network user id and password. Therefore you are authenticated to the web server.
ASP.NET now sees the <identity impersonate="true"/> tag and uses your identity for all further network requests.
Now, the web app makes a connection to SQL server.
If the SQL server is on the same server (as the web app), then you are already authenticated to the web server (it is the first hop). Therefore, SQL server successfully authenticates you.
Instead, as is normal, SQL server is on a different server, then the web app tries to pass on your identity to the SQL server service. The service now asks the underlying operating system to authenticate your network id. That authentication FAILS because the call from the web server to the SQL server is a second hop.

The same applies to the network share scenario.

Implications
What does this mean for enterprises? Enterprises prefer Windows authentication in SQL server because it is not passed around easily and is much easier to manage. But SQL server with windows authentication will NEVER work the ideal way for web apps - by passing through the user's identity from the browser to the web server and then the web server to a separate SQL server.
The enterprise faces the following options:
1. Go for SQL server authentication and its attendent issues (such as passwords and user ids being shared by everyone).
2. Go for Windows authentication, BUT impersonate a specific "service" user account in the web app. That is, instead of setting "<identity impersonate="true"/>", set "<identity impersonate="true" username="appserviceuser" password="password"/>".
You could also encrypt this password and thereby protect yourself from attacks.
In SQL server, then, you give this serviceuser account limited privileges.
This will work because now you are not passing in the primary (browser) user's token. Instead you are generating a new primary token (through the password) for a specific account and authenticating with SQL server.(This will also work for network shares).
The big con here is the proliferation of such service user accounts, making them compliant with password reset policies etc. For example if the service user account's password expires after 30 days, suddenly one fine morning your web app will stop working. So you have to go in and change the password every time.
But, this is the option many smaller companies choose.

3. Use Basic Authentication in IIS. This means that your password is available in the web server and the web application can then (programmatically) use an API to impersonate the web browser user.
This is a really bad option, for a simple reason - let us say this particular app is used by the company's executive management. The web developer can log that supplied password to a file or email it to himself and then get sensitive information about the company.

4. Enable Kerberos delegation
This is a complex mechanism - but the crux is that setting a certain set of options in Active directory enables the web server (or any second server in the first hop) to propagate the server's primary token to other downstream servers. This is a preferred option in some places but it does have a downside: similar to basic authentication, the web developer can use the higher privileges of a end user and programmatically do things he/she is not supposed to do -such as access the boss's email account.

5. Use SiteMinder or Micorosft ISA or such a Single Sign On solution. This will usually result in an Agent mediating authentication. Using such a solution, you can basically use Windows authentication all through - for example, we recently implemented a Business Intelligence solution, with SiteMinder mediating. We could propagate from Sharepoint (MOSS 2007) in one web server to Reporting Services in another to Analysis Services in yet another server - no issues. But this is an expensive solution that bigger enterprises use.

So, take the decision to use Windows authentication (for web-browser end users) in SQL server carefully - it may not be a simple choice.

Summary
The simplest way to remember the Two-Hop rule is this: when you supply a password, it is good for one server to another. That is it. For propagating your identity further in a windows network, you need to enable delegation or collect the password and do it programmatically.
It is important to note that this is not a "limitation" in the network - it is a good rule and exists for a reason.

Microsoft Project and Project Plan Creation

2010-02-25T07:36:00.000-08:00

I have seen a few project managers and most technical people have dificulty creating basic project plans that refect reality and are trackable. I learnt some details about creating simple project plans that I explain here.

1. As a first step, enter the project start date in the project information dialog.
2. Enter holidays and work timings in the appropriate dilogs.
3. Enter a resource list and their availability (50% etc)
4. Add a column called "Work" to the column list. Work is the actual timing for completing a task - it is different from the duration (which is a default column). If a task take 16 hours and two resources work on it, then the Work is 16 hours and the Duration will be a single day of 8 hours.
Always ente work and never enter duration. MS Project will manage duration itself.
5. Start entering the tasks, the Work and the Resources for a task. Do NOT enter dates (start date and end date) when you enter tasks. Dates will be automatically calculated when you level at the end.
6. Tasks can be nested. More on task ordering later.
7. Once you have finished step 5, enter predecessors for the tasks. There are some rules here. NEVER enter predecessor task numbers just to adjust dates. If your dates are wrong, that means your project task arrangement is wrong.
If three tasks under a task heading are done by the same resource(s), then do NOT enter predecessors for each of the three taks. Leveling will take care of the dates correctly.
Again, I repeat, only enter predecessor information if a task actually, in real life, depends on the other task being complete, and is done by a different resource.
8. Now, click on the Level button in the Manage Resources dialog. You HAVE to level a project plan. Many people are afraid of leveling because it seems to mess up the dates - but leveling never goes wrong. What must be wrong is the way you arranged your project tasks or predecessors.
9. If, after levelling, your dates seem wrong (such as different tasks for the same resource starting the same date), do not turn off leveling. Make sure the predecessors are correct. Avoid the temptation to manually enter dates in the mpp.

Project Task Arrangement
In one of my recent projects, we had a Extraction, Tranformation and Loading (ETL) project plan for a business intelligence project. Data is moved from a Temp database to a Staging database and then to a Final database. We were moving data from 20 different files.
The initial project plan simply bunched together the Temp database tasks under one task heading and the Staging database tasks under another. This made the project task predecessors difficult to track.
We had this problem because we saw the project plan simply as a task list instead of reflecting reality. Such a project plan will be useless to track and will quickly grow out of sync with the "real" project status.
After mucking around, we changed it so that each of the 20 files had a separate task heading, with three tasks: Temp data transfer, Staging data transfer and Final data transfer. This led to easier tracking and capturing the natural dependencies between the tasks.
So, be creative in project planning. Do not just dump in your tasks. A big advantage in proper project planning is the ability to do projections - in case you add resources or tasks take more time. For doing projections, it is important that you have a good baseline projectplan.

Exception Handling in .NET projects

2009-11-21T17:47:00.000-08:00

(Updated below)
Here are a few gudelines for Exception Handling. I learnt some of this from the book ".NET Framework Guidelines".

Here are a few questions: Whenever you write a method in a class, is it a good practice to wrap the code in a Try..Catch..Finally block? For example, let us say you are writing a function, GetProductsByProductId. Should thismethod have a Try...Catch..Finally block?
The answer is No. It should have a Try..Finally block, but the Catch is not necessary for every method.
Do NOT catch an exception unless you have a specific reason for handling it. Put a global exception handler and let such exceptions bubble up and be caught in that.
As a corollary, do NOT ever catch the general exception. That is, do not do this:

try
{
...
...
}
catch(Exception ex)
{
...
}

Exception handling needs some thought to go into it. Have a Catch block only if you have a specific reason for handling a specific exception.
One example is this: let us say you load configuration from a file. If the file is not available, you assume certain defaults. In that case, you can handle the "FileNotFound" exception and take an alternative route.

Throwing exceptions
Generally, do NOT catch an exception, wrap it in a custom exception and rethrow it. Let us say a method is not getting the correct parameters. Then, use the .NET exception "InvalidArgumentException". Do not write a custom exception and throw it.

Custom Exceptions
Custom Exceptions are exceptions in your application code, derived from the base Exception class. They can be created for a few purposes. For example, let us say you are handling concurrency checks in the db layer. In case of a failure, you can throw a custom exception. What is the purpose of a custom exception? Usually an application framework handles unforeseen circumstances through a custom exception so that program flow on higher layers can perform useful actions based on such exceptions. Such exceptions are NOT a substitute for well designed interfaces that return correct values.

Service Layer Exceptions
As a best practice, any service methods should ideally return well designed Error objects instead of relying on Custom Exceptions. Web service methods are system boundaries and should not rely on exception handling for propagating errors.
So, handle general exceptions and log them at the service entry. Then return specific error objects from all methods - such as ones with an error code, message (or message id for localizations) and error type.

TryGet Methods
There are a set of methods such as int.TryParse which simply handle failure through a return boolean value. These kind of methods are conventionally written starting with Try.. such as TryGetConfigValue. These methods need not suppress ALL exceptions. They handle an exception and return an error value only if the primary purpose fails. If you have a TryGetConfigValue method and there is an exception getting the config value, that should be suppressed. But such methods need not suppress all exceptions. Any exception unrelated to their primary purpose CAN propagate.

Update I:
My friend Mr.Adrian Gonzalez suggested a few changes to the above post. I am adding them as edits here:

1. I mentioned above that a try..finally block may be necessary in many methods. The purpose of the "finally" is, of course, so that any cleanup code can reside there. Remember that the finally code is ALWAYS executed.
This used to be the place where developers would close database connections or close filestreams. But, .NET has a much cleaner way of closing costly resources: any object that implements the IDisposable interface can be cleaned up the following way:

using (SqlConnection ObjCon = new SqlConnection(Helpers.GetConnectionString()))
{
//do something
}//connection object is closed and recovered here

An instance of any class that implements IDisposable will support the above syntax. Much cleaner than closing the connection after checking its state in a finally block.
Further, do not throw an explicit exception from a finally block.

2. One of the few places that it is ok to capture a general exception (catch(Exception ex)) is when you want to log an exception at that point. For example, developers do this at a service method's root. That is fine.
When you RETHROW an exception, this is good:

try
{
...
}
catch(Exception ex)
{
throw;//preserves call stack for logging
}

This is bad:

try
{
...
}
catch(Exception ex)
{
throw ex;//does not preserve the call stack for logging
}

3. Always include and copy the inner exception when creating a custom exception. Always log the inner exception, if one is available.

4. I mentioned service methods above. In a WCF based web service, there is a way to include an exception as part of your contract. This is called a FaultContract. This has two benefits - you allow specific exceptions to bubble up to the client, in a strongly typed fashion. The client knows to handle them appropriately, even though they are on the other side of a serialization boundary.
The second benefit is that you can add some specific details about the exception back to the client.
For more details about faultcontracts, refer here: Fault Contract Sample

Radius Search, Http Screen Scraping and String to Temp Table

2009-09-27T02:40:00.000-07:00

In the last few days I have been assisting on a friend's website. I have here some code snippets and information I learned.

Comma Separated Id List to Temp Table
Some times you have to do a search on a set of values. Let us say the user is searching through a list of zipcodes for someone close to them. You can write dynamic sql to pass in the list of zipcodes. But then you have to grapple with table security and it will not be fast enough.
Instead, the standard method is to do this:

1. Pass in list of zipcodes as a comma separated varchar string.
2. Create a temp table with a zipcode column
3. Parse the passed in string so that it populates the temp table.
4. Now, join the temp table with the core tables to execute your query.

I have below a code snippet for this:

ALTER PROCEDURE [dbo].[Test_Sproc]
@IdString varchar(1000),
@numIds int
AS

CREATE TABLE #zip (zipcode varchar(5) NOT NULL);

DECLARE @count int, @index int, @start int, @id varchar(5), @length int, @previndex int
SELECT @count = 0, @start = 0, @index = 0, @length =0, @previndex =0
WHILE @count < @numIds
BEGIN
SET @index = CHARINDEX(',',@IdString, @index)

IF @index = 0
BEGIN
SET @length = LEN(@IdString)- @previndex+1
END
ELSE
BEGIN
SET @length = @index - @previndex
END
SET @id = SUBSTRING(@IdString,@start, @length)
INSERT INTO #zip (zipcode) VALUES (@id)
SET @count = @count + 1
SET @start = @index + 1
SET @index = @index + 1
SET @previndex = @index

END

SELECT u.UserId, u.UserName
FROM [User] u
WHERE u.ZipCode IN
(SELECT zipcode FROM #zip)

DROP TABLE #zip

Radius Search
I found that there is a requirement to do zipcode based searches in a geographic circle. Let us say you want to find some restaurants in your neighborhood. Your application lets you search restaurants in a widening circle - such as within a circle of 5 miles, 10 miles and so on.
For this purpose, you need a database of restaurants with address and zipcode. But you also need a zipcode database. The only way you can do a radius search is by getting the lattitude and longitude information.
So you need two components to perform a radius search:
1. A zipcode database that gives you lattitude and longitude information also. Such databases are available commercially for around $100.
2. You need a component that can take the lattitude and longitude and figure out the radius. Such components in Java and .NET are available commercially. You can also build them pretty easily.
Using these you can build a radius search for your users.

Parsing a webpage
In a hurry, I also needed to parse a certain web page for some information, such as a date. If I got the webpage contents in a string I could use regular expressions to parse it. Here is a simple code for getting the contents of a webpage, in C#:

string str = String.Empty;
string url = "http://www.yahoo.com";
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
using (HttpWebResponse response = (HttpWebResponse)req.GetResponse())
{
if (response.StatusCode != HttpStatusCode.OK)
{// throw exception
throw new Exception("Could not get response from web pageUrl");
}
// Read Content
Encoding enc = Encoding.GetEncoding(1252);
using (StreamReader responseStream = new StreamReader(response.GetResponseStream(), enc))
{
str = responseStream.ReadToEnd();
}
}

At the end of the this code, str contains the entire front page contents of yahoo.com. You can then use Regular Expressions to parse it. The above code requires the namespaces System.Net and System.IO.

A New Model used in Rich Internet Applications

2009-08-26T03:00:00.000-07:00

For the past few months I have been working with an Ajax based Rich Internet Application. The architecture in this app is very different from the architecture used in traditional postback-based ASP.NET or JSP applications. This article explains that architecture.

Traditional n-tier Architecture
In traditional layered architectures, the focus is on creating a reusable middle tier in a .NET, Java, PHP or RoR platform. There are a couple of patterns to do so:
1. Create a database layer that accurately represents your business model. Then use an "Active Record" pattern to represent table rows with objects in the middle tier. This means, typically, that there is a class per table. Foreign key and primary key relationships are captured as associations in the middle tier. Each column in a table typically has a property in the middle tier class.
When you use Active Record, your middle tier is a close representation of your database structure.
2. The other option is to create a Domain Model as a representation of your business model. Then map from the domain model to the database using Object Relational Mappers.
In both these options, the use of the middle tier for reusable business logic is a given. Enetreprises usually stay away from business logic in the front end, because enterprises do not usually trust Javascript for expressing business logic. A few reasons for this are: the lack of strong typing, difficulty in automated unit testing, lack of intellisense and strong editor support.

Rich Internet Applications (RIA)
Many applications use Ajax. Usage of Ajax does not classify an app as a Rich Internet Application.
RIAs are typically "single-page" apps - Applications that load a single page with Javascript content and then do not show postbacks (browser refreshes).
Think about a Flash based business application. The Flash movie clip loads additional components on demand. Any communication is using XML and is perfromed with web services in the middle tier.
RIAs, somehow, feel like the "correct" way of developing applications on the web. The client (browser) and the server have a clean separation based on data contracts. Development teams can reach the holy grail of working independently without stepping on each other's code.
Flash and Flex provide a way to deliver business applications with a good user experience. But they are proprietary and there are enterprises that shy away from them because finding developers who code Flex used to be difficult.

Instead enterprises go for HTML/Javascript based RIAs. These have a set of unique challenges:
1. Browser compatibilty - you need a library like JQuery or Prototype to code uniformly across browsers.
2. Component based UI - it is difficult to create UI components in Javascript. Several libraries provide pre-packaged components for grids and trees. But creating your own custom component is still a challenge.
3. Loading additional content and separating dialogs - UI frameworks handle this using iframes. But there is no standard way of handling dialogs. You need to come up with your own framework for that purpose.
4. MVC implementation - the client requires an implementation of the Model-View-Controller pattern so that the server calls and client side data are not tightly coupled with the UI. You may require a custom event pump for handling user actions and messaging the controllers.
Implementing MVC in the client leads to easily maintainable and reusable code. The Flex world have an accepted model called the Cairngorm framework. I am working with a Javascript adapted implementation in my project.

The big question in this case, is the location of business logic.

The Middle-Tier Goes Away
If you end up creating a business model in Javascript and communicate via services with the middle tier, you now have three different places to model the business - the database, the middle tier and in the browser client.
Instead, in my current project, we use the middle tier as a gateway. It simply delegates to stored procedures on the database end. The "data-specific" logic resides
in the sprocs. The UI logic is in the client. there is nothing in the middle tier.
I have heard of a few other businesses using this kind of model.
The biggest relief in this case is that we go from custom XML trees to relational data directly. There is no O/R mapping. The sprocs use XQuery to translate from XML to the relational tables. The transactions are all in the database.
What we have ended up doing is remove the Domain Model from the equation.

Working in this model has raised some deep questions. There are reasons the Middle Tier is the preferred stage (physically) for a business model:
1. It is close to the database; there is less cost to round trips.
2. It provides an object oriented, richer language for expressing business relations.
3. It provides encapsulation much better than available in the database layer.

But creating a full-fledged domain model in our case is redundant because we have a rich Javascript model to handle most of the logic.

From the n-tier model, this new architecture takes us towards a heavy client and a heavy database with a thin middle tier sandwiched in between.

A concept in Facebook security

2009-07-10T23:12:00.000-07:00

I have worked with a couple of Facebook apps since the platform was introduced. When working with developers, I have noticed that an explanation of a simple facebook security fundamental is missing. That fundamental concept helps in understanding what is possible and what is not possible very well. You do not have to check the API every time to understand it.
You may have uploaded files to a webserver. Have you noticed that there always needs to be a file upload button in HTML? You cannot automatically upload a file using Javascript. The reason is obvious - certain actions require user permissions in the browser. Unless the user explicitly clicks a button you cannot take some actions.

The Facebook security model has a simple concept like this:

The User Context
Let us take the case of getting a list of the user's friends. This is private information for the user. Therefore, the API call that gets a list of the user's friends needs the user's action to execute.
That is, you CANNOT get a user's friends unless the user is online in Facebook and explicitly clicked a button. If, on the other hand you DO want an app to perform this functionality, of getting a user's friends WITHOUT the user being online (say in the case of a batch job), you need to ask the user for an OFFLINE permission. This permission is not granted by default - it is an extended permission.
Facebook API calls are actually callbacks. When user clicks a button, Facebook sends the request to the third party app's website. That app then calls the facebook API back to get some information. Logically, even though an app is in a different web server, we can view the action from clicking a button, to the application's response as a single thread. This thread has the user's context (in the form of a TEMPORARY session id).
Thus, if you looked at the API call for getting an user's friends it does NOT usually take a input user id. It takes the current session key.
In the absence of a current session key, you can pass in a input user id, but the app needs to have the extended permission.
This applies to most API calls and it is a very easy concept to understand. In effect, Facebook is restricting you from using the Facebook user id as if it is a general database query identifier.

Scenarios

Let us take an example scenario your client comes to you with: User A visits the application canvas page and schedules a message for all his/her friends. That message should be sent only at midnight of User A's birthday. So the client wants you to store the user's request and then execute a batch job at the scheduled night to send the message to all friends. Can you do this?
The answer is you can't - without extended permissions. When your batch job executes at the middle of the night, it needs to get a list of User A's friends. For that it will call the Freinds.get API above. That API call WILL fail - because you are NOT getting the user's friends in the user's context. The user is neither online nor did they actually execute an action in the facebook app page. So, without the user's context, your call will fail.
Faced with this situation, I have seen developers suggesting workarounds - like storing the user's temporary session key in the database. My advice is GIVE UP! That restriction is there for protecting the users and you should not violate it. If you try to find a loophole, you will keep searching.

Let us consider another scenario - user enters a comment about a movie in the Flixster app. It goes to a moderator, who checks if the comment is appropriate. Then the moderator clicks the approved button - the client now wants the user's comments to be posted in the user's feed. Is this possible?
Again, the answer is no (unless the app has extended permissions). The moderator is NOT the original user. Think about it - would you like it if another user in facebook started posting entries in YOUR feed? You wouldn't. Therefore facebook does not allow (by default) an app to make entries in the feed outside the user's context.
Remember, this is different from the app itself asking you if it can post an entry in your feed. That will be allowed - because the app is still ACTING in the user's context.
Do you see the difference between the above two scenarios?
Let me explain again - let us say you take a Quiz. At the end of the quiz, the app prompts you if it can make the results available in your feed. This is fine, there is no violation of security. That is because the app is still having your session key available. You are online and you click a button. Facebook now allows the app to post an entry in your feed. The reason is because this is NO different from you posting it yourself. There is no extended permission needed for this case beyond the default permissions granted to an app.
What Facebook will NOT allow (without special permissions) is the ability of an app to post an entry to your feed OUTSIDE your user context - such as from a moderator's session.

In other words, Facebook allows apps to perform actions that the user can do when he/she is online. It does not allow apps to perform such actions on an user's network when the user is not online or from another user's session.

So, all that you have to do when a client comes up with a requirementis to ask yourself whether the action is within the user's context. If not, check extended permissions. Ideally an app should not have to operate with extended permissions. That is bad design.
Check out about Facebook's extended permissions here. You will see, for example, that an application can send Notifications when user is offline, but it CANNOT send Emails to user's inbox when user is offline (such as, through a batch process).

More information on Facebook Authorization is here.

Roles and Permissions - Authorization models

2009-07-04T23:39:00.000-07:00

This post tries to capture the two different models in place for authorization in web applications.
Usually every web app has at least two different roles of users accessing the application. If you take a business application, like a timesheet managemenet system (my favorite example), you have employees who access the system to enter their time and task details. You have supervisors who access the system to approve such timesheets, and also enter their own task details.
The problem of identifying who the user is (by means of passwords or certificates) is the domain of Authentication. But, once the user is logged in, we have to allow or deny the user their various actions - this is the domain of Authorization. Authorization efforts are usually distributed all around the application code.
In the above timesheet example, an employee who is NOT a supervisor should not be shown task details of other users. This is accomplished in code by not showing the list of other users to a non-supervisor.
So, at some point in the code we have to make a decision of showing a link or not based on the user's authority.
If the application has to be really secure, the authorization will extend horizontally and vertically.
Horizontally, in another area of the code, let us say that the supervisor sees the timesheet of a person reporting to him/her. The "Approve" button for the timesheet is in that screen. The code that displays this button should still check if the user has the authority to approve or not. That is, even though a non-supervisor will NEVER gain entry to view another user's timesheet, the code for approval should still assume that they can and disable the approve button at that point.
Vertically, when the operation to approve a timesheet is actually called, the code should again check for the current user's authority to approve timesheets. That is, disabling buttons and links on the UI alone is NOT sufficient for authorization.

Thus, we can see that the code has to check for authority at different points in a given user's context. There are two different models for managing such authorizations.

Role and Permission based Authorization

For simplicity, let us consider a case of disabling or enabling the "Approve" button for an user. Remember the example extends both horizontally and vertically.
Authorizing a user based on their role assumes the following narrative:
A user who is in a supervisor role is accessing the Approval screen. Should the "Approve" button be enabled or disabled?

Authorizing a user based on Permissions assumes the following narrative:
A user who has the Approve permission for the Timesheet object is accessing the Approval screen. Should the "Approve" button be disabled or enabled?

Readers familiar with .NET would see that the first method corresponds to the Principal.IsInRole() method in the IPrincipal interface. There is no equivalent for permission based authorization in .NET.
One of the most important challenges in implementing these models is that they should be configurable or data-driven. That is, an user's roles should be available in the database. Similarly user permissions should be available in the database, in the case of permission-based authorization.
Let us consider these models in more detail below.

Role-based Authorization

The Users of a non-trivial web-based system will be available in the database. During initial design, the roles throughout the system will be clear - in the Timesheet system above, there are supervisors, non-supervisors and system admins. System admins are a hidden role in most systems.
So, we need a Role table to store the system-wide roles along with their identifiers. The Role table is usually small and only has a few rows.
Next we need a table mapping Users to Roles. This is usually a link table, say, User_Role_lnk.
Note that users in this model have multiple roles. This is usual in most systems - Roles are not mutually exclusive. In the Timesheet example, a supervisor also needs to access regular features like entering timesheets for him/herself.
Thus, typically a User has multiple Roles.
When the user has been authenticated, the system loads their roles in some object and makes it available in the current thread's context. So at different points in the code, developers can check for role authority this way:

if(currentUser.IsInRole("supervisor"))
{
enable approve button
}
else
{
disable approve button
}

This is a simple model, but it can get complicated some times. Since the user can belong to multiple roles, for some code, you may need to do this:

if(currentUser.IsInRole("supervisor") or currentUser.IsInRole("admin")
{
do something
}

In a system with many overlapping roles, you could easily have many such if..else conditions, thereby creating code that is difficult to understand or maintain.

Permission-based Authorization

The argument for authorization based on permissions is as follows:
At every point when you need to check authority, we are concentrating on what role an user belongs to INSTEAD of concentrating on what the code itself needs permissions for. If we flip the foreground and the background, then we should focus on what permissions a block of code needs.
The ideal, in the case of Permission-based authorization, is code that looks like the following:

if(user.hasPermission("Approve") on object Timesheet)
{
enable Approve button
}

This generally creates more maintainable code. But the details of the data model can be complicated.
Remember that we need more entities to define permissions. Let us look at them, in light of the pseudocode above.
We need a User entity as before.
We need a list of Permissions
We need a list of Objects
Note that the goal is to be data-driven. Usually we call the objects that we are "protecting" (such as the Timesheet object in the above code) as a Resource. In one implementation I saw, the Resource name along with its fully qualified namespace was used to represent the protected Resource in a "Resources" table.
So, unlike the Role-based model, we actually need a list of Resources in the code that we are protecting.
At first, this seems difficult to understand. In case of disabling the "Approve" button, what is our Resource name? What Resource in the database should it translate to?
The "Approve" button protects the Approve operation (which is probably setting a flag) on the Timesheet object, which is an instance of the Timesheet class. So, the Resource we are "protecting" or "seeking authority for" is the fully qualified name of the Timesheet class.

So till now, the table model consists of Resources, Permissions and Users.

Roles in a Permission-based system

But when we are actually assigning permissions, we still would like to assign it to a group of users than a single user. The User inherits those permissions simply because they are a member of a group.
Sounds like a Role, doesn't it?
So, the permission based model needs to include the idea of roles for ADMINISTRATION, but when the Permissions are being evaluated at Runtime, the roles work only in the background. The developer still checks only for permissions. The roles are usually not exposed in the Security API.
Let me explain with an example:
We have a user A who is associated with two groups, G1 and G2. G1 has the Approve permission for the Timesheet object. G2 has the Create permission for the Timesheet object. Therefore, user A gets the Create AND Approve permissions at runtime.
This can of course quickly lead to another scenario - if G1 has Approve permission for the Timesheet object and G2 explicitly has the Deny permission for the Timesheet object, what does the user get?
This problem is not unlike the problem faced by an operating system when you login to Windows in an Active Directory network. Of all the conflicting permissions from different groups you belong to, which one wins?
This is a detail of implementation - it is upto the business analysts to determine if Deny trumps all permissions or not. You just have to be aware of this possibility when implementing a permission based system.

Implementation of a Permission-based Authority

As a summary, a permission based system requires a custom implementation. It requires you to take into account group conflicts. It also requires you to enumerate (in the database) Resources that you are protecting.
Building a GUI administration system for Permissions is complex, compared with simply populating User-Role link tables.
But this model is very common in business applications and produces more maintainable code.

Establishing an Internet Presence

2009-05-03T00:00:00.001-07:00

I wrote about the administrative details of starting freelancing here. In the below article, let me talk about a few guidelines for getting clients.
When you plan to take the freelancing route, as I mentioned in the above article, be clear about your role: A freelancer in IT is usually a programmer (or graphic designer). You need to be ready to program and be hands-on.
You are NOT starting a business - for now. You are a one-man show initially. Do not be over-aggressive in going after clients. Do not commit to more than you can do. All the rules that apply as a programmer in a project applies to a free-lancer too.
There are certain possibilities for getting clients as a freelancer - let us consider these:

The GetAFreeLancer route
Here is a link to a set of freelancing websites for coding: 85+ Tools & Resources for Freelancing
When you start, it is tempting to look at getafreelancer or other sites and assume that you can bid lower than everyone and get projects. This route is completely packed with a million other programmers all over the world. The clients are mostly cheap and try to lowball every bid. They have no idea about the true value of programming and may not pay at the end either. I have seen people who ask for an Email campaign system for $200. It is ridiculous, and I would advise you to stay away from this route.

The NRI route
As I mentioned in the previous article, if you have been abroad for some time, maintain contacts with your immediate friends. This does NOT mean that you go after any contact like an Amway rep. Stay within your comfort zone but let people know that you will be available for freelancing. Open a LinkedIn.com account and keep it current.
In my experience, if you let people know that you are going back to India, they will approach you with offers. In this technological age, remote working is pretty easy and so feel confident about going into freelancing.

The Long Hard route
If you are in India, have not had time to cultivate contacts abroad, you can still get clients. But it is a long haul. You need to prepare from now. Most Indian programmers with a talent and passion for programming would be in the situation I describe below:
1. You will have had some years of experience (3 or more) in a services company.
2. You will have worked with foreign clients. You would have communicated with them via email,IM; you would also have interacted with them personally when they come over or you go abroad on short visits.
3. You know that there are a set of "soft skills" - writing grammatical English with some structure; how to organize your thoughts and write emails; and you understand a few foriegn accents. If you are not skilled at these, you atleast know that you have gaps and try to get better.

The method I suggest below takes a long-term view. It may not yield results immediately; but there is no question that it is a clean route to take.

Note: Most services companies have non-solicitation agreements. Based on those agreements, you cannot usually work with direct clients of your company for a time-period. So, PERISH the thought of freelancing directly for your companies' clients.

Establishing an Internet Presence
1. Register a web domain name with a good hosting provider. For now, go cheap. Think a bit about your domain name. Remember that you MAY grow to be a business at some point, so create a professional sounding name.
2. Make sure that you get a hosting provider which offers platforms with the technology you are most comfortable with - such as .NET framework and SQL Server if you are Microsoft; or PHP and MySQL if you are in PHP.
3. Google (and other providers) offers email services under your own domain. For example, if your site is www.mygreatcompany.com, then you can get email services for @mygreatcompany.com. Most hosting providers give you a way to map from the domain name to a third party email service such as Gmail. Use that and create professional email ids such as sales@mygreatcompany.com and an email address with your name.
4. Some hosting providers give blogging packages for a small fee. That way, you can host your blog in your site itself. Or you can try the route I chose - I have a blogspot technical blogging address (http://ramsnotes.blogspot.com). I link my articles from my website (http://www.alphazonsystems.com).
Having a blog is important. Having some backend such as .NET or PHP with database support is also important, even if your site consists of html pages only at this stage.
5. Have a graphic designer design a decent website. Develop it and make it available for public.

At this stage what is the content strategy for your website? How do you get traffic?
You have to create content based on your goals. Your goal is to be known as a good technologist. This website will be seen by your potential clients. They will NOT take a decision solely based on how good your content is, but it is definitely ONE of the factors. You are trying to gain popularity in the internet.
The best way to create content is to write blog articles. These articles can be of certain types:
a) Explaining a concept such as XML parsing or concurrency checks in a O-R framework.
b) Explaining a tip such as how to trace HTTP calls with Fiddler while browsing from Firefox.
c) Exploring a new technology such as Microsoft Silverlight or Adobe Flash Collaboration Service.

I have written a detailed article on the idea of blogging here. You need to learn to write well. Apart from writing articles, provide free code. People will always visit sites with free code. For example, if you wrote a business app based on Silverlight as a proof of concept, provide the code as a zip file, linked and downloadable from your website.
If you create good content, then there are two advantages from it:
1. Obvious one - people will visit your site; they will link to you. It serves as online credntials.
2. Subtle one - for every article you publish, send the link to your clients, friends and contacts. Put the link in LinkedIn.com, facebook and so on. What that does is it REMINDS your contacts that there is a talented guy/gal out there who is available. They will keep being reminded by your work. Let us say, instead, that you send an email every month telling everyone that you need contracts. people will not react favorably to that. But sending the blog link is a very subtle reminder.
I think this is the best way to use the internet as a marketing tool for personal work.

One big question is how far you can market yourself while STILL working for another company. If you look at my website, alphazonsystems.com, I am explicitly marketing myself. But I ONLY did it after I left my last job. Before that, the site was available and so were the articles. But I did not cross the line and ask for work for myself. I think that is safe. Maintain a website and post articles, but don't let your company feel threatened.

This internet presence you create will take time to accumulate. Over time people will link to you, download code. But you may not get actual clients who ask for services. That is fine. There are other indirect reasons it will help you.

The Website as Credential
As I said earlier, you cannot directly approach your clients. But your clients have friends and their own network. THAT is the network you can tap into. It is one level separated from your own network. If you impress the people known to you, they will refer you and remember you when someone is looking for a freelancer.
That network does not know you personally. But your website and the work that has accumulated in that website over a period of a couple of years will serve to convince them to trust you.
If you are passionate about technology and want to freelance anytime in the future, it is IMPERATIVE for you to establish an online professional presence. Start the process NOW.

So, again, the steps to create an internet presence are:
1. Create a website
2. Create a blog in your website or separately
3. Start writing a couple of articles
4. Popularize your articles by linking them in your website; linking them from your IM; Facebook; LinkedIn; and send emails with the links to everyone who likes technology in your circle.
5. Write frequently
6. Create demos in your website with free code downloads.
7. You can also write articles in online tech journals and link to them. It requires professional writing skills but you should try.

Back to my original point, you require this kind of long haul approach to seeking clients for freelancing. Particularly if you do not have many contacts abroad.

Being Self-Employed in IT

2009-04-24T20:37:00.000-07:00

I have been self-employed doing freelancing programming work for the past 6 months. Let me describe how I got into it and a few administrative details.
Taking the plunge
There is a difference between being self-employed and running a business. If you plan to run a business, you can face exponential growth or risky shutdown. Freelancing is not the same thing, even though there is pressure for you to expand. (Note: Do NOT freelance if you plan to get loans. Banks do not give out loans to freelancers)
If you are in the USA and planning to come back to India, or if you are already in India but have enough contacts in the USA, freelancing may be a good idea for you. Let your contacts know that you plan to freelance. Your contacts will need some credentials to justify hiring you. For that purpose, register a domain and have a basic website - if only with html. Put your resume outline in that resume and highlight your work. If you can write well (with grammar and good content), then link your blog articles in the website. Have a set of email addresses registered under your domain. My own website is here:http://www.alphazonsystems.com/
Freelancing works well if you have a stable contact who trusts you and vice versa. Companies may not trust just an unknown freelancer in another country. You need a champion at the client. It would be ideal if he/she can coordinate payments and also help you get set up for starting. The gain the companies have is, of course, lower rates.
So, I would advice you to cultivate potential clients if you are abroad and planning to come back. If you are already in India, do not hesitate to contact your friends abroad.
Technically, you will generally need a VPN connection to the client. You will need a high speed internet connection.
You will also need to be self-motivated to work by yourself. I maintain a timesheet even if the client does not ask for it. I have regular calls and status updates sent to them - and that keeps my work planned.

Infrastructure and Administration
Freelancing, of course, needs the following infrastructural and administrative elements:
1. You need your own computer, preferably a laptop. Make sure you have a warranty.
2. Get an UPS for electricity failures.
3. You may not need an office space, if you have enough space at home. Otherwise, get a small enough office space, close to home. Don't plan a long commute or spend too much money on decorating the office.
4. Get a webcam.
5. Open a Skype account with SkypeOut (ability to call international phones). Start with a $10 SkypeOut credit. For the SkypeOut payment, get an online receipt printed for your tax purpose.
6. Get a good set of headphone with a microphone.
7. Get atleast two physical file folders. Label one of them for your business account in the bank and the other for your taxes. (I will come to banking and taxes later)
8. Since you will spend time at home, make sure you have a decent broadband internet connection. Atleast 512kbps will be needed.
9. Get a good computer table and chair. Make sure your working location has good airconditioning. This is important, because if you are at work, the AC is takend care of by a company. If you only have AC at bedroom at home (as most middle class households do), then you will find yourself trying to work from the bedroom all the time. That is not a very good place to work from. Plan for a second AC room.
10. Buy an external hard disk with atleast 160GB for backup. Have a bunch of DVDs for storing your important files. Have a USB flash drive with 4-8GB capacity. If possible, sign up with a service like Windows folder share or SugarSync for an online backup of your programming work.
11. Backups are VERY important. Take a full system backup atleast once a month. If your client uses source control, check in your files regularly.
12. Get a copy of Norton Anti Virus with Internet Security. Run LiveUpdate once a week. Keep running Windows Update every week or have it run automatically. Keep your computer well patched and do not have any anonymous shares. If you use Vista, run under the UAC (User Access Control) mode all the time. You cannot afford a virus infection.
13. Get a laser printer; they are not that costly now and will help you out.
14. For ALL of the above purchases, whether online or not, get a receipt. You will need it for tax purposes.

Founding the Company
In India, there are different kind of company structure - proprietorship firm, partnership or private limited. For a beginning freelancer, a proprietorship firm is ideal. It ties to a single bank account and you can use your personal PAN card for filing taxes. (You can still hire people under a proprietorship firm). Other than the bank account, you do not need any other form of registration. Here are the steps for founding a proprietorship firm:
1. First decide on a name. Make sure the name is also unique for starting a website.
2. Make sure you have a individual PAN card. A PAN card in your name is sufficient.
3. Decide on a bank. Contact the bank official and let them know you will be running a current account for your business. They will give you a set of guidelines for opening a current account.
4. You need rubber stamps. 3 stamps at least - a seal called a "For" seal that you will affix with your signature to documents; a seal called the address seal with your company name and address; and a date seal. All these rubber stamps cost around Rs.600-800.
5. You need letter pads and a logo. If you already have a website designed, you may have a logo. Otherwise design a logo, a letter pad and a visiting card. A graphic designer can do this for you. Get a letter pad printed and the visiting card printed. (The visiting card is not essential).
6. Contact an auditor near your location. Let them know you are starting a company. You would go back to them when filing your taxes.
7. After the bank opens your account, they will send you a checkbook.
8. Usually, if you work with clients from abroad, the easiest and fastest way to transfer money (payment for your services) is to wire transfer. But wire transferring has charges - upto $100 for a single transfer. The charge applies to the person doing the transfer ie your client. They may deduct it from your payment. If you do plan wire transfers, then get the wire transfer details document from your bank. Every bank has a slightly different procedure and there are some details to fill in. Send the details to your client.
9. From a stationery shop get a "Cash Voucher" book and a receipt book. They may come in handy when you document expenses.
10. For invoices, get an invoice template online. Every invoice should have a unique number and will have the amount you will be charging your client. (Invoices are the "bill" for your client. They will make payments according to the invoices.)
9. With a bank account, stamps, letter pad, PAN card, invoice template, cash voucher book and (if possible) a website, you are all set to go.

Payments and Taxes
Before you negotiate with your clients, you need to know this: it is better to be on a hourly work basis than on fixed-bid work. If your work is such that the clients engage you for a long term with a steady hourly payment, that situation is ideal. Please note that a commitment for an engagement long term does not mean much - the client can cut you off at any point with payment for services till that point.
So for hourly work, what would be your "take-home"? Let us say you bill $100 every hour (unlikely, but it is easy for calculations. Being in India, you would probably only bill less than half that amount).
There is no service tax for dollar exports in software coding services. So, your taxes would come to 20%-30% (depending on your slab). Let us say you started work in January; If by end of March, you earn less than 5 Lakhs, then your slab would be 20%. But, this is assuming you had no other income from the previous April to December. You are taxed for your entire income from April to March.
So $30 of every $100 you earn would go to the government ideally. But there are a few expenses that you can claim to lessen your tax load.
(Your US client may want to know if he needs to tax you in the USA. Because India and the USA have a dual taxation treaty, your income will NOT be taxed twice. )
The expenses you can claim are:
Capital expenses (all your initial investment such as buying the UPS, printer, headphones, stamps and so on)
Location rent (if you work from home, only 50% of your rent is eligible for deductions. If you have an office, your entire rent is eligible)
Transport (if you commute to work; have a car or two-wheeler)
Water
Phone and broadband
Stationery (not much use in IT)
Entertainment (such as lunch with your client at the Taj)
You are a proprietorship firm - you can hire people. You do NOT have to deduct tax for them. Your payments to them count as expenses.
Your LIC, PPF and other savings still apply.
Keep a receipt of ALL your expenses in a physical file. Auditors advise to keep a note of credits and expenses in a excel file.
You have to pay advance taxes twice a year (March and September) and file returns in June. For advance taxes, you would sit with your auditor and go over all the expenses and income. The advance tax can be paid in a local bank such as SBI or HDFC. Keep the advance tax receipt safe.
When you file returns in June, you will include the advance tax receipt and then adjust your final payment according to actual figures. At that point, you need all the proof of your expenses. The Income Tax office may ask you to come in person and ask you to submit the proofs.
If you receive foreign exchange payment, then do NOT forget to get the receipts from your bank called the FIRC. You need it for submission to the Income Tax office.

How to Plan Future Work
The biggest problem you will face as a freelancer is insecurity. Your contract may end at any time. It is tempting to take additional contracts to stave off this danger, but then you may end up having a hard time managing two contracts and clients. So, you tend to try to hire someone. But then you have to face all the problems that comes with trusting a third party - he/she may not like to work for such a small shop. they may disappear any day. They may ruin your reputation with clients.
So, it is a dilemma. I have avoided this by deciding early on never to hire other full-time people. I also decided that I will not take additional full-time contracts. Sure, my position is risky in that case - I actually lost a couple of contracts because I refused to work full-time for them. But that is fine for me.
One key struggle psychologically is NOT to consider yourself a budding star entrepreneur. It is very tempting, particularly with all the propaganda about entrepreneurs to consider yourself a special sort of human. I remind myself constantly that I am simply self-employed and not running a business. It helps.

Concurrency Checks and Scenarios in business apps

2009-03-21T00:51:00.000-07:00

This whole blog is based on some notes I collected on concurrency checks in a web application.
While designing a persistent object framework, or designing a web application (say, a Task Assignment and Management app), one of the tricky questions is the handling of concurrency while saving data.

Definition
For the purposes of this essay, a concurrency issue comes up when user A loads data from the app database; user B also loads data from the app database. User A makes some changes and saves back to the database. Now, when user B also makes changes and tries to save, how should the application handle it? User B’s data has become stale (because user A has changed it AFTER user B loaded it).and therefore his save may result in a wrong decision.
In our task management app, let us say User A sees a task with task id 111. This task has a description, status and a person it Is assigned to. It may also have task notes.
The database table looks like this:
task_id task_desc task_status task_assignedto

It has the following values for task id 111:
task_id task_desc task_status task_assignedto
111 Fix error Pending User A

Now user A sees the above record in his screen and decides to change the status to Completed. He changes the status and saves the record.
Meanwhile, at the same time, user B, who is a project manager is also looking at task id 111 in her screen. She thinks the task has been pending for a long time. So she changes the status to “Escalate to Supervisor” and saves the task.
But, her data is stale – user A has already changed the status to Completed. If she now escalates, it would only result in confusion all around.
This is a common scenario in applications (web or desktop) ; what just happened was a concurrency issue or a “save collission”. There are different techniques to handle save collission and we will discuss a few here.

Concurrency Check Approaches
1- Last Save Wins approach
A simple application may just choose to avoid concurrency altogether – the designer may think that two users editing the same record is unlikely. Sometimes, even if two users edit the same record, the designer may choose a “Last Save Wins” approach. In this approach, there are no concurrency checks in the database layer. If user B saves after making changes, those changes overwrite user A’s changes.

2 - Concurrency Check using all original data
ADO.NET and some frameworks have a default concurrency check based on all the columns in a table. In the Save stored procedure, the query will look like this:

Update Task
Set task_desc=@ desc

task_status = @status
task_assignedto =
@assignedto
Where
task_id=@id

And task_desc =
@original_desc

And task_status =
@original_status

And task_assignedto = @original_assignedto

The stored procedure with the above query takes the original data that an user (User B) loaded. Then it checks if ALL the columns are identical ie, that no user has changed the data after User B loaded the data. If ANY of the columns have new data, the above query fails to update. The stored procedure gets the number of rows updated and if the number is zero, it raises a concurrency exception.
In a web application, the original data has to be cached in the client side or in session storage in the web server. Every save sends back the original data.

Now, this approach helps in the case of auto-generated queries – you do not have to code different rules for different tables (and objects). There is another, simpler approach for such concurrency checks.

3 - Concurrency check using Timestamp
Every table in the database should have a “LastModified” datetime or timestamp field. When User B loads her data, the LastModified timestamp goes back to her session. Now, when User A modifies the same task, the timestamp is modified for that task record. [This requires that any update to the Task table should always also update the LastModified column].
Finally, when User B tries to modify the same record, the stored procedure runs the following query:

Update Task
Set task_desc=@ desc

task_status = @status
task_assignedto =
@assignedto
task_lastmodified = getdate()
Where
task_id=@id

And task_lastmodified = @original_timestamp

Let us stop at this point and consider the effects of a concurrency error. What should happen in application logic when a concurrency exception is thrown from the database or persistence layer?

Concurrency Usecases

The answer depends on the business user’s requirements. Generally the use case is that the application:
- notifies the user (User B) that her save failed because someone else had modified the data after she loaded it.
- prompts for a refresh of the record(s) so that User B sees the new changes, thus losing her changes
- Or asks User B if they want to save in spite of the concurrency error. At this point, in complicated scenarios, User B may actually be shown the modified record(s) in a new window User B now has enough information to make a decision to overwrite another user’s modifications or cancel the save and lose her changes.
For handling these more refined usecases, in which the user B is shown the modified record, you may have to return the new recordset from the save stored procedure.
Note: - Notice that in the above description, there is a possibility of multiple records being saved and retrieved. Till now we have been considering a single task record, for the sake of simplicity. In an non-trivial application, there would be an object map that is getting saved back. An example is when all tasks assigned to a certain person are being modified for moving a deadline. In a persistence framework, such a transaction would be handled as a parent-child relation and a series of stored procedures may be called. Handling concurrency in such “units of work” is more complicated – we will return to that later in this essay.

An Ideal approach to Concurrency
The above three approaches are standard in most business applications but they are not ideal. The reason frameworks deal with such approaches is because it is easier to generate code for stored procedures when you adopt a single consistent approach for concurrency issues.
In reality, a business application may need specific, tailor-made concurrency checks for each object. Let us consider the Task Management scenario above again.
Let us say User A loads the task 111. User B also loads the same task. Now User A modifies just the description of the task and saves it back. Now, when User B saves her data, should the stored procedure throw a concurrency error? It depends but in general users may not prefer a concurrency error for something as simple as a description change. In this case, User B’s changes may be allowed to overwrite User A’s description change.
Thus the stored procedure should check for specific columns instead of all columns based upon business rules set by a business analyst. This is painful and tedious, which is why you see general approaches such as using the last modified timestamp for any column updates.
Note:- Please keep in mind that an user could also have deleted a record while another is editing that same record. It is ideal that the concurrency check in that case specifically inform User B that their changes were not saved because the underlying record was deleted.

Note 2:- Please also keep in mind that multiple users may have changed the record in between. Thus, the change may be more than a single update.

Thus, an ideal approach to concurrency issues in an application would follow the below checklist:
- collect concurrency rules for important tables and columns in those tables.
- create stored procedures that would allow certain columns to be overwritten while other column changes should throw concurrency errors.
- While throwing a concurrency error, attach a list with the offending fields’ user friendly display names; the new values; the users who made changes in between; the different times when changes were made (an entire change history after load).
- prompt the user to ovewrite individual changes.
- When the user chooses to overwrite, run the save procedure again and overwrite specific columns

It is just impossible to do all of the above – the closest systems that come to handling concurrency checks in such detail are version control systems such as Visual Source Safe or SVN. I have never seen a business app do all of this.

The most common approach I have seen uses the below checklist:
- create a last modified timestamp column
- any update would modify that column
- that column would be loaded along with the table data
- on a save, the stored procedure would check if the timestamp column has changed. If yes, throw a concurrency error.
- The application would handle the concurrency error by informing the user that the save failed and prompt them to reload the data.

Parent Child Object Graphs and Concurrency
If you are saving multiple records that are associated with each other, at the same time, then there are two common scenarios:
1 - Middle Tier Transaction – multiple stored procedures are called in the context of a middle tier transaction. If three records have concurrency errors ( that is they are stale and have been modified since load), and another ten records go through without concurrency errors, then the decision is again upto business.
Business may prefer that a single concurrency error in any one record causes the entire transaction to fail; or they may prefer that the ten successful saves not be rolled back. The ideal scenario again dictates that the user be prompted with a list of records that were not saved. The user chooses to overwrite or not.

2 - Database Transaction – in the case of rich internet applications, sometimes the middle tier simply serves as a proxy and all the batches of data is routed to the database, in the form of Xml. The database stored procedure parses the Xml and creates new records, modifies existing or deletes. All of these happen (generally) in the context of a single database transaction.
I have worked with such systems and handling concurrency is easier. In the case of an error, you assemble a new Xml document with the changed records and send them back to the browser client.

Rich Internet Scenario
Let us consider such a rich internet scenario:
User B sees a tree control of security groups. In the client, User B drags and drops a few branches, modifies some descriptions and deletes a few groups. Then she save the tree structure back.
Meanwhile User A (as usual) has done a few changes himself and already saved the structure back.
What should the concurrency rules be?
When the stored procedure gets User B’s Xml data, it parses the Xml and saves individual records. While saving each record, it checks a bunch of things:

- If this is a deleted record, then mark all child branches as deleted and so on recursively.
- if this is a new record, then check if its parent exists and has not been deleted by another user. If the parent branch has been deleted, mark the new record as having a concurrency exception.
- If this is a existing record that has been modified, then check if its immediate parent exists and has not been deleted. If the parent branch has been deleted, mark the modified record as a concurrency exception.
- If this is a existing record that has been modified, then check if its immediate parent or any of its ancestors have been modified (by using the last modified timestamp). If yes, then mark the modified record as a concurrency exception
- Description changes do not cause the last modified timestamp to be changed.

Following these rules, the stored procedure creates a set of records that were not saved and returns them to User B. The application prompts User B on the next course of action – overwrite (which may not make much sense); or cancel your changes.
The above is just one set of algorithms you can use to implement concurrency checks in a hierarchical structure.

Merge Issues
One big problem in rich internet applications is that the user could be modifying a lot of records. In case of a concurrency error, they may not want to lose the changes they made. One complicated usecase is to do a client side merge of the new data in the server along with User B’s changes. The merge rules may be complicated sometimes but may also be simple.
Consider that a set of security groups is being modified by a system admin. The admin is modifying security groups but is also adding a few new ones. Now, when she saves, a concurrency error occurs. The admin may choose to merge her data - in which case, the tree control in the client will still contain the new records, plus the reloaded data. Setting a merge policy would require much custom code.

Summary
When you start designing an application consider the concurrency checks needed. Design your stored procedures upfront with the checks built in. Design your middle tier, client and stored procedures considering the overwrite and merge rules. The closest model to think about concurrency is in terms of version control systems.

Ext JS, Scriptaculous - Horizontal Portals

2009-02-09T20:54:00.000-08:00

The Ext-js library has a portal example here. If you play around with that, you can see that the "portlets" move vertically when you drag and drop. It is a frequent requirement, instead, that the portlets dispose horizontally.
Imagine a horizontal panel as in the image:

There are two challenges to modify to horizontal:
1. Create a horizontal layout with a scrollbar
For this, the standard technique is to do the following steps:
a. Create a outer div with a fixed width, say 700px. Set "overflow:auto" in the outer div.
b. Create a div within it. Identify this div as, say, innerDiv.
c. In Javascript code, let us say you have 5 portlets, each of width 200px. Set the innerDiv width to 1100 px in code. In Ext-js, you do it thus:

var containerPanel = Ext.get("innerDiv");
var aItems = [];//this array should contain the list of portlets
var iMinWidth = 200;
var iWidth = iMinWidth*(aItems.length) + 60;
var strWidth = iWidth.toString() + "px";
containerPanel.setStyle("width",strWidth);

So, the key idea is that the inner div's width is set dynamically, depending on the number of portlets contained within. This also works if you have to add portlets on click of a button.

With the above steps you can create a scrollable container, with portlets laid out horizontally.

How to create the portlets
For this, I followed the portal example in Ext-js. I modified the sample.js to contain the following code for my horizontal portal sample:
The portal.html contains (for me):

<div id="divWork" style="overflow:auto;width:700px;" >
<div id="innerDiv">
</div>
</div>

The sample.js contains:

Ext.onReady(function(){

// create some portlet tools using built in Ext tool ids
var tools = [{
id:'gear',
handler: function(){
Ext.Msg.alert('Message', 'The Settings tool was clicked.');
}
},{
id:'close',
handler: function(e, target, panel){
Ext.Msg.alert('Message', 'The close tool was clicked.');

}
}];

var aItems = [{
title: 'Another Panel 1',
tools: tools,
html: Ext.example.shortBogusMarkup
},{
title: 'Another Panel 2',
tools: tools,
html: Ext.example.shortBogusMarkup
},{
title: 'Another Panel 3',
tools: tools,
html: Ext.example.shortBogusMarkup
},{
title: 'Another Panel 4',
tools: tools,
html: Ext.example.shortBogusMarkup
}];

var containerPanel = Ext.get("innerWork");
var ul = containerPanel.createChild({tag:'ul',id:"ulWork"});
var iMinWidth = 200;
var iWidth = iMinWidth*(aItems.length) + 60;
var strWidth = iWidth.toString() + "px";
containerPanel.setStyle("width",strWidth);

for(var x=0; x<aItems.length; x++)
{
var seqid = x;
var sStyle = "float:left;width:180px;height:100px;display:inline;margin:5px 5px 5px 5px;";

var li = ul.createChild({tag:'li', style:sStyle, id:"li_"+x });
var rd = new Ext.Panel(aItems[x]);
rd.render(li);

}

Sortable.create("ulWork", {tag:"li",constraint:"horizontal", onUpdate:function(){alert("Dropped");}});

});

Thus, within the inner Div, I have a "ul" and add the portlets as "li"s one by one.
The last line, is meant for drag and drop - as covered next.

2. For drag and drop of the portlets
I completely bypassed Ext. Since the above list can be considered a sortable list, I simply created a Sortable object on the ul.
This single line enables drag and drop. More details can be found here.

So, with the above, you can create a set of portlets that are horizontally laid out. Although other portions of our application uses Ext-js, this particular sample uses scriptaculous and regular Javascript to accomplish key functionality.

Internet Archiving - who owns my data?

2009-01-04T10:05:00.000-08:00

(Update below)

Something has been bothering me about the internet (or more precisely, the collection of websites I use in the internet).
As an user, I have a broad range of options available now for publishing my content (such as blogs, images, video). But I do feel paranoid about certain qualities in the current internet.

Let me describe my situation - I write comments in several online discussion forums (such as rediff.com). I write blogs in DailyKos.com and blogspot.com. I upload videos to youtube.com. I also write comments in other people's blogs, whichever platform they may be. I save my bookmarks in del.icio.us.
Now, I value all this content I put in the web. For example, in several debates you come up with a new way of looking at something, an effective reply to a "talking point", a key piece of data that shuts up everybody. I am not talking about other people's content - I am specifically talking about content I myself put on the web in different websites. This is the age of User Generated Content and my content is distributed across different websites.
There are a couple of problems that I face with this distributed content:
1. How can I aggregate all my content and get updates when someone replies to me or links to me? This is a problem that RSS solves. I will not elaborate on this here.
2. How can I collect and provide a kind of catalog of my opinions in all these different websites? Let us say that in the near future I seek admission to Harvard. Is there someway that I can provide a collection of all my valuable content to Harvard so that I can be credentialized? Looking into the future, we can expect a new generation to start creating their identity online by teenage and thus leave a trail of their work and impressions (in whatever format) across the interner as they grow older. How can someone maintain this digital trail and leverage it?
The point is this problem has always existed even before the age of the internet. For example, if you wanted to collect the complete works of Einstein you went to every university he ever went to and searched libraries and archives. Some of these archives are digitized now, but nobody came up with a solution for an easy mechanism to package up your life's work. It was close to impossible in the pre-internet world to have a centralized collection of all of one's life's work.
But, this problem is solvable in the digital age - the facebook, orkut, myspace generation is going to have access to internet most of the time. Fifty years from now, it is possible to expect that a person's life's work can be determined by a biographer or an anthologist merely from the digital world.

So, what prevents me from getting a digital collection of my own work distributed across the internet, right now?
I can, of course, prepare a set of links with my rediff.com comments, my dailykos blogs, my blogspot blogs and my youtube videos. But that is all I can do - the websites reserve the right to invalidate these links at any point of time. In fact, twenty years from now, many of these may have switched off their servers and gone home.
As an internet contributor, the core problem I face is this - the rich data that is part of "my" internet, is not owned by me. It is owned by at least fifteen different websites. The same problem goes for everyone using the internet.
When I write an article in DailyKos, I want my article, along with the comments (which provide context) to be available for posterity. But I have no control over when they may "retire" the article or when Markos closes it down.
In theory, this is no different from the problems faced by preceding generations - if you are a newspaper columnist, you took paper cuttings of your column; probably photocopied it and kept it at home. That is all you could do.
I think we, in the internet age should demand more though - because more is possible now. For example, taking a printout of a webpage with my article is not good enough - because someone could be commenting on that article this minute; and I don't want to lose that context.
With their myriad ways of annotating, commenting and extending our content, the websites of the internet have made my content richer, more contextual, more centralized than in the pre-internet world. Youtube, blogspot, rediff have all made my contributions richer, but because of that it is more important that I be able to catalog and archive that content.

The tension here is between two poles - the websites have enabled me to contribute and reach a broad audience. For the survival of their business, they want me to keep coming back to their pages. So the data stays in different forms in different websites. I, on the other hand, would like to extract my data (in some format) and keep it in a set of archives so that the data is available for posterity. I am worried that all my valuable contributions will be gone some twenty years later.

I may sound paranoid, but I do care about the longevity of my thoughts - I think everyone who contributes in the internet does.

We should not allow what happened in the past centuries - there was no centralized publishing, so much work of enormous value was gone in a short time because the medium (like parchment or paper) perished. In this age almost everyone with access to the internet can publish their opinions and share their knowledge. It is all searchable. We should leverage the advantage of the digital medium and come up with a solution for extracting our data (even for a fee) and create some standards for extracting User Generated Content.
We also need a standard archiving solution that is not tied to any particular website. For a fee I should be able to store my data in different servers that are part of the meta-internet.
By the way, check out this browser add-in - http://www.iterasi.com. It provides a way for extracting any webpage for your personal storage. But the storage is still owned by iterasi.com - I think we need an internet archiving project that is more community-driven.

Update I
Refer to what happened to Soapblox in this article.
The content of several blogs running on the Soapblox platform was almost wiped out by hackers. Several years work could have been lost. This is why we need an open source extraction and archiving system.

Add-In for Javascript Development using Visual Studio

2009-01-01T02:55:00.000-08:00

I have created a Add-In for Visual Studio (2005 and 2008) that helps Javascript development. I developed it for my own use since I code now in Javascript extensively. But it is extensible enough that I have created a project in CodePlex.com for it - here: http://www.codeplex.com/CodeOrganizer.
It helps navigate Javascript; creates a very cludgy "intellisense" window; and also transfers code fragments to different regions. My purpose in creating it was to avoid scrolling back and forth in big Javascript files. When you edit CSharp and VB, you can easily navigate straight to functions and have intellisense do half the work - but Javascript editing is more difficult.
VS 2008 has very good Javascript intellisense , but it has its limitations. In our projects it does not work at all. I wrote to Microsoft about it; they are working on an improved engine apparently. Meanwhile, the add-in assists me. It is also invaluable in navigation.

If you try it out, I welcome feedback. You are also welcome to use the template to develop your own add-ins for any productivity improvement. The code is free for distribution under the MIT license.
You can view a demo (not very good quality in video) here:

The Implications of Live Notifications

2008-11-15T03:02:00.000-08:00

There is a trend among web 2.0 websites to add live chat or notifications to their websites. I have had the opportunity to work with a couple of them – one specialises in a product that provides such live chats to websites; the other is creating notifications and IM-like behavior for users browsing their pages.
Facebook of course added live notifications – you can see which of your friends are online and send them messages. Google has Gtalk purely on the web. This post analyzes the problems with such notifications.

Structure of Live Notifications

A chat or notification feature typically requires polling the web server. The user's browser has to go back to the server to check for any messages. It also writes the user's messages for other people. Such polling is done using Ajax webservice calls so that the user does not notice it.
Take the example of GTalk (Google's online chat). There are several webservices in that application:
1. when user logs in there is a notification POST saved in their data store so that user's contacts can see him/her online
2. when user logs in there is a "GET" to get the user's contacts who are online
3. when user types a message for a contact it is POSTed to that contact
4. when the contact types a message, the user's client GETs new messages addressed to him/her.
5. when user logs out there is a notification POST so that his/her contacts do not see him online
6. the user can also change their online status - that is a POST

Of these, items 2 and 4 require the user's client to constantly poll the server at a system-configurable interval. If this interval is high (say 2 minutes), the user gets frustrated. If it is low, the server takes a huge hit.

Implications of Live Notifications

Adding such notifications to a website requires careful analysis. The reason is because of the constant polling and writing data, Live Notifications increase the workload of a public website.
In a website there may be hundreds of online users - they may be going through different pages. The number of online users at the same time is called simultaneous user count. But the webservers themselves may be taking a lower hit - since many of the users may be reading content and otherwise browsing through a page, the number of concurrent users will generally be lower than the number of simultaneous users. Therefore a website can afford to have an infrastructure that supports a certain number of concurrent users. A set of concurrent users translates to a set of threads executing or waiting at the same time in a web server.
But, if the website introduces Live Notifications, the usage pattern changes dramatically. The constant polling makes the number of concurrent users close to the number of simultaneous users. Thus even if the the number of users in that website has not changed, the servers may be in for a beating.

This may not happen in a desktop IM client like Yahoo! Messenger - desktop IMs can afford peer-to-peer communication and therefore will not affect servers to the extent that a web model does.

Solutions for Live Notifications

The bottleneck in a Live Notification feature is the database server. Since conversation histories have to be stored, and every contact around the world should be able to send messages, database writes are unavoidable. They go up heavily in such a scenario. But the real problem is database reads. These may cost a server and a normal running server may crash if you add Live Notifications.
The web server is usually stateless in such interactions and thus scaling by adding more would work.

There are generally different levels of investment needed for companies deploying such features.

1. If your online (simultaneous) user base is low (say 2000-5000 users online at a time), you may be able to get away with a stateless webserver talking to a dedicated database server.

2. If your online user base is in the tens of thousands, you cannot afford reads to hit the database. You would need a solution in the middle-tier like memcached. This would keep the user's messages in the middle-tier and enable polling to retrieve new messages from the memory instead of the database. You will need a set of servers talking to a memcached server. You will also need failover for the memcached server.

3. If your online userbase is in the hundreds of thousands, you may need clusters of servers in the middle-tier and a complex addressing scheme so that messages can be picked up from a server. This is heavy investment in infrastructure, maintenance and engineering.

If your user base is low, then consider sticking to solution 1.
Before directly stepping into solution 3 ask yourselves this question - is adding such notifications a critical feature needed by my website? It is a specialty area.

Javascript Libraries

2008-11-06T05:59:00.000-08:00

When you are designing an application there are several paths regarding the choice of UI:
1. If the application is internal to an enterprise, you can choose a desktop application. Mostly instead you choose an application delivered through the browser to avoid installations. Even through the browser, you have options like .NET Smart Clients that are opened in the browser. You have the option of Flash/Flex or Silverlight - a consistent, rich UI experience with Drag/Drop is possible in this environment. You can also choose traditional JSP or ASP.NET applications with postbacks.
2. If the application is facing public users, then most of the above options are eliminated - you have to go with Javascript/HTML/CSS - because it is the paradigm of the web.

In both the above cases, there is a reluctance to choose Javascript to build strong UI - the reason is that Javascript programming is not very type-safe. Editors are hard to come by. There is little use of intellisense. It is still seen as just a "scripting" language. In the browser, there are still cross-browser issues in DHTML, Ajax and so on.
I wrote a post sometimes back about Object Oriented Javascript - mainly talking about simple OOJ concepts. But in the past few years, almost every month someone releases a Javascript library. Javascript libraries such as prototype are immensely popular for a few good reasons:
1. Cross-browser issues are transparently handled by these libraries
2. They provide standard controls, layout managers and a great programming model.
3. They also provide means of extending such controls using OOJ.
4. They provide a custom Event model.
Thus, developers and architects dealing with decisions on web based delivery should necessarily consider Javascript libraries. Since there are hundreds of them, you should be aware of a few considerations while choosing one:
1. The libraries size matters - prototype.js is small, but UI effects needs libraries like scriptaculous. These can add to the size of the HTTP response - but can be easily managed by web server compression and caching headers. The libraries also come in a "minified" form for deployment.
2. Some of them come with complex controls that you can then extend to create your own. They are proper frameworks. You may choose them based upon your UI needs.
3. Select a library that has a popular user base. JQuery, Dojo, scriptaculous, mooTools, and the now-popular ext-js all have a good user base. Do NOT expect, though, very high level documentation.
All of these libraries provide services such as:
1. Cross-browser DHTML
2. Ajax
3. Common Controls such as trees, accordion
4. Drag and Drop and other effects that work cross-browser
5. JSON and XML support

So, if you are considering an internal or external facing website consider Javascript programming using one of these libraries. It makes life a lot simpler. You gain all of the services by including the necessary javascript files in your header.
(Also knowing one of these libraries provides a fast path to knowing others).

To start exploring, download one of these libraries and go through the tutorials. It does not take much time to figure out (particularly if you know OOJ).

Qucik Note on Economic Downturn

2008-10-09T08:10:00.001-07:00

If you are in the IT Services industry in India and your clients are international, you may be familiar with the downturn and impending job cuts. Many of you may be facing a recession for the first time. I have a few suggestions from my experience. Think about them:
1. Your company maybe churning and you may be assigned to different projects. You may be asked to take paycuts. Your location preferences may not be respected. Companies usually crack the whip during a downturn. You may be outside your comfort zone - working with managers unknown to you, and you may have to re-establish your credentials.
All this causes much anxiety and sometimes prompts irrational decisions such as lashing out at the work place; taking a break from work; going on long leave; or even trying to switch jobs.
My advice is do not make emotional decisions based on fear or anger - particularly at this juncture. It is hard to think rationally and it is possible that you may feel ill-used or neglected. It is also possible that these feelings are purely psychological. You will not be able to judge this by yourself. Therefore do NOT make life-changing decisions, such as leaving your job; taking a break; plan a career switch; plan an academic sojourn; or anything like that. Think carefully about your decisions for a few days. Discuss with friends and family. As far as possible do not plan a change.
2. Save, save and save money. Do not take long vacations; defer decisions to buy costly items.
3. If you are laid off - remember, whatever society or your former employer tells you, you know your true value. These are bad times and people may lose their jobs for being in the wrong department at the wrong time. There are forces much beyond our control at work - do NOT blame yourself or fall into self-pity.
4. Have a strong support structure at work and at home. In tough times, friends need to support each other. Have atleast a few friends with whom you can share your fears and inner-most thoughts. That will be more helpful than sitting and brooding.
5. Do not antagonize anybody at work for perceived insults. This is not the time to be touchy about not responding to emails etc. Everybody has their own problems and you never know when a contact will be useful.

What is OpenSocial?

2008-09-24T21:19:00.000-07:00

The OpenSocial standard is currently at version 0.8. Already, it has been implemented by many social networking sites such as Friendster, Hi5 and Orkut. What does the standard mean?
In 2007, Facebook.com came out with an API and a platform for delivering widgets. A widget can be a word game, or a set of friends sharing movie reviews. Facebook followed a fairly complex model to render such widgets, and let them access facebook's social API.
But facebook widgets work only with facebook. They have to be basically rewritten to work with Orkut or other social sites.
Later in 2007, Google released the OpenSocial standard. That standard provides a spec for social networking sites (on the one hand) and widget writers (on the other) to implement. If sites Orkut and Friendster implement the OpenSocial standard, then a widget that works in Orkut will work with Friendster with no rewriting.
As with any standard, OpenSocial is based upon an abstraction of the services provided by a social network. Google also leveraged its existing Gadget spec (Google Gadgets) and added OpenSocial as a "feature" of gadgets.

Google Gadgets
Google gadgets are based upon the gadget spec. The gadget writer creates a gadget definition (which is in XML). This definition contains (among other things) Javascript code that can be called from the browser.
In a typical scenario, the gadget definition is written and is available in a public url - http://www.mygadget.com/mygadget.xml.
The gadget container is the host that is going to display the gadget. That site can be http://www.myhost.com. When you hit the host page, it will send a request to the gadget server, passing in the url of the gadget xml (in this case,http://www.mygadget.com/mygadget.xml). Google has a freely hosted gadget server called gmodules.com. There is also an open source project called Shindig that provides the code for a gadget server. The gadget server does the heavy lifting of translating the gadget definition to HTML that can be rendered on the browser. Among other things, the gadget definition provides localized "message bundles" for getting localized strings. A gadget can ask for different "features" from the gadget server.

Thus Google gadgets can be run in your website, provided you have a gadget server available.

Open Social
Google built Open Social as one of the features of the gadget server.
In the google gadget example above, there are three servers involved - the gadget definition host,the gadget server and the gadget container. The gadget container, in social networks, is a social container. For example, the social container knows who the current user is, his profile information, who his friends are and what his activities are. Abstractly, these are modelled as a set of social API. The social container renders the gadget and then provides social API services to it.
Take the case of Orkut - Orkut contains social data about you. In its profile pages and canvas pages, it renders applications or "widgets" These widgets are rendered using gmodules.com as the gadget server. The widget definitions are hosted in different third party web servers.
Orkut provides its social API to the widget by exposing Javascript functions.
Because these Javascript functions and their namespaces are a OpenSocial standrad, the same widget can work in any OpenSocial compliant website, such as Hi5 or Friendster.

If you are writing a widget you should refer to the gadget spec and the open social spec. Let us say a sample widget allows you to send messages to all your friends in a particular network. That widget will call a OpenSocial API function to get the list of friends and show them to you. It will call a different OpenSocial API function to send messages to the friends' inbox. Both these functions are available via the Javascript API.

A sample gadget definition is below:
<xml version="1.0" encoding="UTF-8" >
<Module>
<ModulePrefs title="List Friends Example>
<Require feature="opensocial-0.7"/>
</ModulePrefs>
<Content type="html>
<![CDATA[
<script type="text/javascript>
/** * Request for friend information. */ function getData() { var req = opensocial.newDataRequest(); req.add(req.newFetchPersonRequest(opensocial.DataRequest.PersonId.VIEWER), 'viewer'); req.add(req.newFetchPeopleRequest(opensocial.DataRequest.Group.VIEWER_FRIENDS), 'viewerFriends'); req.send(onLoadFriends); };
/** * Parses the response to the friend information request and generates * html to list the friends along with their display name. * * @param {Object} dataResponse Friend information that was requested. */ function onLoadFriends(dataResponse) { var viewer = dataResponse.get('viewer').getData(); var html = 'Friends of ' + viewer.getDisplayName(); html += ':

' + person.getDisplayName() + '

'; document.getElementById('message').innerHTML = html; };
gadgets.util.registerOnLoadHandler(getData);
<script>
<div id="message">
</div>]]>
<Content>
</Module>

If you are a social network, and you want to implement OpenSocial, then you have to do the following:
- Download, install and run Shindig (there is a PHP and Java version). That will be your gadget server - you can, of course, implement your own gadget server, by going through the gadget and open social spec.
- Implement the Javascript API by making calls back to your social network database from the browser. Implement this in a library and have it included in your gadget rendering page
- Implement a set of hosting pages for the gadgets (typically using iframes) so that the request for the gadget goes through the gadget server.
- Implement an authorization scheme.
- Implement a set of pages for searching, adding and removing widgets from a user's profile.

In conclusion, OpenSocial is an abstraction of the social API of a container and the gadget rendering modules of any widget.

How To Blog

2008-09-24T10:41:00.000-07:00

This is my 51st post in this blog. I thought it should be about what I have learnt about blogging.
I will cover the following in this entry:
- How to start a Blog and some technical tips
- Why would you blog? What is the purpose of a blog? Is it necessary to start one? Is it a fad? What do we write about? How often?
The opinions are strictly based on my experiences.
I am not going to define a blog and all that. Please read Wikipedia's excellent entries on that here.

How To Start a Blog

What you need is a personal blog that you can use. This personal blog will
- allow you to manage all your "posts" or entries
- allow you to create new posts
- allow you to manage "templates" for your blog - that is your background color, layout, top banner and such things.
- allow you to manage comments in your own blog, so that you can delete them if offensive.
- manage history in your blog by archiving old blogs

Wordpress.com, LiveJournal.com, Blogger.com and many other sites allow you to do most of the above. The assumption here is that you want a public blog. Some blogging sites also allow you to create members only blogs. You can create a blog and invite a set of people to view it. Noone else will have access. Corporations also have blogs in their local network for knowledge management.
There are also several blogging sites that are domain specific - for example, DailyKos.com is an American political blogging site. Your audience are mostly looking for political commentary in such a site - they may not be interested in your love poetry.
The choice of a blogging site is, thus, guided by your interests. If all you are interested is in writing about Microsoft technologies, then there are several sites specific to that purpose. Some bloggers (like me) maintain several blogs at different sites. That makes it slightly difficult to manage your posts but targeted audience reach may be better.
One important consideration is multi-language support. There are several sites (including blogger.com) that provide support for Unicode - thus if you want to write in Tamil, you need not create your own site from scratch.
Most blogging sites allow you to embed images and videos. You can effectively build a corporate style article using a blogging site. Check for such features.
Most sites provide steps to create a blog that are really simple. Generally,
- create a username, password and a profile. Depending upon your nature, you may want more or less detail in your blog profile. Be aware that people across the internet, who do not know you, may like/dislike your opinions and may come after you. On the other hand, in case of a technical/professional blog particularly, you may want people to know you for your skills. Choose carefully what information you reveal.
- choose a template. In my blog, the orange color top banner was picked up by me, along with the layout. I have a different color in my technical blog. When you choose a template, I beg you, please do not pick white text on a dark background. It is tempting, but the way computer monitors work, your site will hurt people's eyes.

That is it. After this, you can go to your dashboard and start typing a blog any time.

Apart from the blog itself, you may be interested in knowing how many people visit, and also make your blog searchable. My articles on my Coorg trip, Ilayaraja's music and others still attract traffic mainly because people reach them through searches. How do you measure number of people visiting; what is popular; how to get attention? How do you know who links to your articles?
I follow these tips:
1. I downloaded and installed Google Toolbar in my browser. It shows the page rank for my blogs. That kind of gives a rough idea if you are even "seen" by Google.
2. I registered my blogs with Technorati.com. There is a set of steps to do this, including adding some code to your blog template. They give detailed instructions on how to do this. Technorati clearly tells me who else in the blogging world links to my blog. It also shows authority and genuinity.
3. I registered my blogs with Google Analytics. They also have a set of steps and some code to add to your blog template. You can log into Google Analytics at any time and see data about the number of people visiting your site, which pages they are interested in, their geographical spread, the time they spent in an article and if the traffic came from direct hits, referring sites or search engines. It is an amazingly effective tool to gauge your effectiveness as a blogger.

Why Blog - The philosophy of blogging

Should you always blog in this day and age? We see people around us starting blogs all the time. They do it for different purposes. In my case I have some key motivations.
- I have been trying writing since I was 6 years. I have tried my hand at fiction, writing letters to the editor in local newspapers, tried participating in writing competitions. Basically I would rather be a writer and do it for a living. Before blogging I used to send emails to a bunch of people with my thoughts on different subjects even though my audience hated those emails. Thus, when I discovered blogging, I primarily thought of it as another tool for reaching a larger audience with my writing. Thus I did NOT start writing with blogs. I was always "blogging" in some form, even before the internet. If you have never been interested in writing, blogging may not help you. It will only make a person feel guilty about not maintaining a blog frequently.
- Even if you are not too interested in writing, blogging (for a professional) is a great way to build an online presence, and showcase your knowledge. It is one of the best resume formats. A frequently updated blog link makes people want to recruit a person. In my job searches, I send out cover letters with links to my blog articles - it is unusual, but credentializes you before you meet a person.
- Writer or not, a blog is a good way to atleast hone my marketing and presentation skills. You can experiment with a larger audience than you can imagine. Particularly for technical leaders and aspiring technologists, it is a good way to organize your thought process.
- One great reason to blog is its usefulness as a knowledge management tool. One of my earliest posts was about a few recipes for cooking. They were simple recipes, but they are available in the internet wherever I go. I refer to Praveen's blog about how to enable fiddler for Firefox all the time. I do not have to remember these details. You can blog about the little tips and tricks you learn in your professional and home life and then have them available to help you later.

Now, note that in all of these cases, there is a clear self-interest to blogging. There are people who have a different view - they claim that popularity of their blog is not their concern. They claim that the number of comments their blog attracts does not interest them. When someone mentioned such a near nirvana state in the Chennai blogcamp (2006) there were responses in agreement and disagreement. It is hard for me to adopt such an attitude about my blog. I shamelessly pimp my blogs to my friends and contacts. I want my views to be known - that is why I use analytics and technorati to measure my performance. I value feedback and discussion on what I am writing about.

Some tips for interested bloggers

1. What do you write about?
In my experience, when I think of some interesting angle, I brood over it for a day. Then I sit down and write a blog article for an hour. But I have noticed that I am constantly looking for such angles. I select some ideas, I reject others. Other bloggers have talked about a kind of urge (restlessness) you feel if you have not written a post for some time. All of this, of course, depends on time being available (more on that next). I write for my technical blog when I feel it is necessary to store some detail or process some knowledge.
Thus, in general,
- you can write opinion or commentary posts
- or you can write an informative, observation based post
Similarly,
- you can write long article sized posts (such as this one)
- or you can write shorter posts
Opinion or commentary posts attract people's attention. My Ilayaraja vs Rahman post still is one of my top picks. People will feel either for or against and they will comment on it. Of course, your opinion should be authentic, and have some fresh angle. Otherwise, you will not get repeat visitors and a regular readership.
But informative posts are also valuable - they will help you manage knowledge and build expertise. They will also provide more durable value to your readers.
I usually alternate between these. For example, this post you are reading is meant to be informative.
The core idea is to keep looking at any knowledge you acquire as a potential blog post.

2. When do you write?
I do not allocate any special time for writing and I do not stick to a schedule. Bloggers like Glenn Greenwald and Digby blog daily multiple times. But they also make money from blogging. Most of us amateurs cannot blog at that frequency.
The problem is to capture readership and make them come back for more. A frequency of once a month is insufficient for maintaining loyal readers.
There are many of us who start blogging ferociously and then just stop at a certain point. Then we feel guilty about stopping. There is no point in feeling guilty about not blogging - the readership is low, and most of the blogs do not attract traffic at all. It is hard to keep your spirits up under these circumstances. Also, as I said earlier, writing may not be your idea of fun. If so, there is no need to feel guilty.

3. How to attract initial readers?
I read an advice for frequent blogging, with one long article atleast a week and multiple shorter posts during the weekdays. Hard to do, but does attract loyalty (provided the content is good).
Send the blog article link to all your friends, family (when you feel ready about the content). There may be habitual forwarders among them and they will spread the news.
Have your freinds with popular blogs to link your blog.
If you have MSN or Yahoo Messenger, put the blog url in your signature. Put the url in your Facebook or Orkut or LinkedIn profile.
And have good content. There is no substitute for good content. Market yourself, but have good content.

4. Some Do's and Don'ts
Do NOT plagiarize - do not steal someone else's content and dump it in your blog. Do not quote an entire article and then add a link at the end. Write original content. Since you are publicizing your blog, having copied content will lead to people having a lower opinion of you.
Do NOT hope to make money out of blogging. Takes a very long time to be established and very hard to make money out of it.
When you finish writing a post, DO re-read it again for spelling mistakes and grammatical errors. They turn off visitors.
Whenever you quote someone's blog or opinion online, DO add a link to that quote. It is professional to do so.

Writing Technical Specifications

2008-08-12T05:05:00.000-07:00

Training technical leads to write a technical specification document is challenging. A technical specification follows a functional specification from a business owner (typically in the process flow). I view a technical specification as a dialog between the technical lead and the development team (inward facing). Sometimes it is a dialog between the technical lead and a technical stakeholder in a client's organization (outward facing). In both these cases, a tech lead needs training on how to go about writing the spec.
Here is a key attitude shift - please do not consider the spec as something you have to write to get the process documents complete. The spec writing process will organize your thought process, and create a narrative for the solution path you choose. It is similar to writing a story - in the structure and the narrative.
Before you start writing a spec you should be clear about the meaning of solution architecture. What do you focus on when you approach a system? At what detail? How important is UML for this purpose?

What is Technical Architecture?

Let us consider the Indian Railways Online Reservation System - http://irctc.co.in. The Railways are going to create an online reservation system and they have invited you as the Architect. What do you do (other than demanding free tickets for the rest of your life)?

There are different views of the system - there is an end state and then you have to get from the current state to the end state. Here are a few views of the system:

a. Users need to see the list of trains, see availability and then book the tickets. How do you get the list of trains and add new trains to the system? You need to see the list of free seats - how do you get that information in a database? Thus there is a data model that you have to create.
b. You have to design your middle tier code such that it can handle tens of thousands of users. That requires caching some of the information that you get every time - for example, the list of trains between two destinations can probably be cached. You have to come up with a object model. You also need to decide about state management (such as session).
c. Your UI has to be responsive and cool - you need minimum HTTP requests and need to deal with popups or overlays. You need cross-browser compatibility. These are frontend design concerns.
d. You have to manage a security scheme that handles different roles - some users may have admin capabilities to add additional routes or trains. You also have to protect your site against cross-site scripting attacks or SQL Injection attacks. So there is a security model.
e. This application must perform well for tens of thousands of users. That means you have to design for performance and consider upfront the performance tests on each module. You also have to plan the number of servers you will require on the website. This is the area of performance testing and capacity planning. You also have to plan for sharing state aong different webservers. This is your deployment model.
f. You have to process payment by credit cards to different banks. This is the area of payment gateway integration. You may need to support multiple banks through a similar interface. It must be easy to add another bank. You also have to make sure that your data is secure (encrypted) if you comply with PCI or other standards.
g. You have to process RACs (Reservation After Cancellation) in a background job. Thus there are a list of supporting back office services that have to run to support the website. You have to identify them and suggest a model (workflow). You have to monitor that these services are working properly and get alerted when they do not.
h. You may have to choose technologies based upon the decisions above. That includes constraints such as your current development staff skills.
i. You need a network model because the application is available across the country to ticketing agents everywhere.
j. How would this system be tested? What are the environments you would use to deploy the system and what are the rules of deployment?

Done? Not yet.

The biggest issue you would face on this project is the rollout plan. One fine morning the application goes live and availability information should be available online for that day and going forward. This means there is a data migration from paper based systems (or a legacy digital system) to the new online system. How does that happen? It is within the architectural plan to come up with a solution for this problem.

Obviously the solution path is complex and you have to handle different views of the system. But the above information is what you will put together in a technical specification document.
What is the level of detail you go to? In High Level Design (HLD) documents (or architectural specifications) you do not need class diagrams unless essential to explain a solution.
You can
- describe a problem,
- suggest your approach,
- alternative approaches
- pros and cons with each approach
and be done with it. This has to be repeated for each "area of concern" as shown above.
In the Low Level Design (LLD) documents (or technical specifications) you would focus on detailed design for each module including UML diagrams.

I have mostly worked in services organizations in which there is a client technical stakeholder - so I usually use the HLD as a dialog between me and the client; and the LLD as a dialog between me and the team. The HLD and the functional spec give a good overview of the project to a development team.

Tips on the Document Creation

i. I always organize tech specs in a conversational style and freely use the terms "we", "I" or "you" - this sounds informal, but helps the narrative.

ii. I do not hesitate to put questions and possible alternatives in the document. It helps later when someone else needs to understand your thought process. I highlight unanswered questions in different versions using an yellow background.

iii.I create a template structure with the following format:
1. Business overview
2. Scope and out of scope list for the project
3. Overview of the document - with a list of key items that will be addressed
4. References - such as functional specs
5. Development Environment, link to coding conventions, tools, IDE, version of frameworks, source control, language to be used
6. Architecture Overview - standard diagram of the system so that people can understand what we are trying to achieve. You can include existing system architecture diagram in the case of reengineering projects.
7,8,... From here on you address specific concern areas. Each should be in a separate section.
I usually create the above template upfront with the concern areas also listed. Then I start drilling deeper.

iv. I do not create diagrams while writing the spec. I mention the diagrams in the document, but I draw them later - so that it does not interfere with my thought process. Usually people take time drawing using Visio or other tools. If you really want to draw it out to get a good idea, draw it on paper or draw it on a white board and photograph it. It does help to have a set of Visio templates available.

v. I usually have the following sections always available in every HLD document:
- Security
- Performance
- Unit Testing
- Health Monitoring
- Rollout Plan

MySql High Availability

2008-08-09T10:07:00.000-07:00

I recently had the opportunity to work on a solution for MySQL High Availability. Since I had no idea how MySQL worked and what High Avail meant, it was a crash course for a week. At the end of the week I had a good idea about the “primitives” in database HA solutions. It was, surprisingly, simple to understand.
High Availability solutions are concerned with database services being up most of the time (or the “five 9” SLA – databases to be available 99.999%). Although the specific solutions vary across different vendors (such as Microsoft SQL Server, Oracle and MySQL), there is a common thread or a set of concepts cutting across vendors.
Any database solution is a service that runs different threads servicing connection requests. This service runs on a physical or virtual server. The service accesses data from a data storage. Usually, the data storage is in the file system. But it can also be part of a Storage Area Network (SAN). SANs are used in large enterprises and allow network devises to be used as local file systems. Either way, the service need not be aware of this distinction. As far as a service is considered, the data storage is local.
A normal production system, that does not have high availability may use a single database server that runs the service which accesses the local file system. If the service fails, someone needs to be notified. The System Administrator (SA) would then restart the service and check what went wrong. This means for a variable time, the database is down and so is the application.

MySQL Replication

For high availability, you necessarily need at least two servers or “nodes”. You can have more than two, but two is minimum. If one server or service fails, the other takes over.
This seems simple – once a server goes down, the SA can have the second server take over.
But there are two problems with this scenario
- The data in the first server needs to be available in the second. If it is not, the second server has older data and in a commercial system, that is unacceptable.
- The application is still pointing to the first server. We need to change the IP of the second server to take the place of the first, so that the application can continue to function. This is called Automatic IP failover

To solve the first problem, MySQL uses the Master/Slave configuration or MySQL Replication. When MySQL is configured, you create on server as a Master and the other as a Slave (there can be multiple slaves). MySQL Master internally takes care of replicating data at frequent intervals to the Slave. Thus, when the Master fails, you have most of the data in the Slave. There may be a small delat missing after the last synchronization, but that may usually be accaptable, depending on the replication time interval.
Note that MySQL handles this replication, not a separate service.
So, at this point, the SA gets notified on Master failure. SA converts Slave to Master and manually switches the IP of the Slave. Application can now start working while the SA fixes the Master.

Note: With Master/Slave, in websites which have mostly read-only access, you can have the Slave handle most of the reads, while the Master handles the reads and writes. This requires an application level change (like changing connection strings in PHP or .NET). Thus there is a level of load-balancing you can achieve.

General parameters for HA

Thus, you can see that running a High Availability solution needs you to understand four important parameters. Ask yourselves these four questions when looking at a clustering solution from any database vendor:
1. How do I get alerted when a service goes down? Most data centers have service-independent Alerting systems that would page a system administrator. There is a Linux service called the Heartbeat that can monitor the operation of any service in Linux.
2. Does the recovery or fail-over need manual intervention? If no, what is the time interval for which the service is broken? As you can see in Master/Slave, the answer is Yes, we do require manual intervention so that a SA can configure the Slave to be the new Master.
3. Is the data available completely after failure recovery? In Master/Slave, we already saw that there is a short loss of data. But this can be resynchronized manually using transaction logs in the Master.
4. How many nodes (servers) do I need to run a HA system? This corresponds to the infrastructure cost. Master/Slave can run with two nodes, while MySQL Clustering needs as many as 5 nodes to run a HA system. Of course this is a tradeoff, because the system recovery in a MySQL cluster does not require manual intervention and can happen in less thatn 3 seconds in case of failure.

There are other options for HA in MySQL such as MySQL+Heartbeat+DRBD and MySQL Clustering. I encourage you to explore these and apply the above four questions to each solution. In fact, you can use these for judging SQL Server and Oracle clustering too.

Insight of the day

2008-08-07T07:37:00.000-07:00

Most of the professionals in this industry spend time proving to others that they are smart. That they can understand requirements or technology without asking too many questions or without showing ignorance. I think many projects would actually be more successful if we were just allowed to say we don't jack about what the client is talking about. Same holds through all ranks - if developers could tell project leads that they had no idea what was expected out of them, it would actually move projects faster - I am positive.

Toondoo Exploration

2008-06-30T11:18:00.000-07:00

Trying out a nice web 2.0 site called http://www.toondoo.com.
I created the cartoon below called Delivery and my wife created the one below called Night Life:

Scroll over the image:

A wakeup call for IT Architects and Leads

2008-06-29T05:19:00.000-07:00

(This article is cross-posted in http://www.photoninfotech.com)
Sometime back I interviewed a senior technical architect for Photon. One of the standard questions I ask is if such senior people have worked in developing or maintaining a public website.
Invariably this question leads me to define a public website. A website that is available on the DMZ for entering timesheets around the world is not really a public website. What I usually mean is a website that does commerce on the internet or more generally has something to offer to random users.
Such a website is different from internal web applications because of their broad usage patterns, traffic, marketing needs, analytic needs, usability and so on.
Anyway, this particular architect said Starbucks contacted his company to develop such a public website for them. He also volunteered that he had suggested using a "smart client" based approach for that.
I was taken aback - "Smart Client" is a Microsoft term used to describe .NET based WinForms applications that run in Internet Explorer. To run a smart client app in your computer, you need the .NET runtime installed.
I asked him if that is what he meant and he confirmed it was. I asked him what kind of marketing you could do on a website that only allowed people who had .NET installed. He had no answer - obviously he had no idea about public websites and it is probable that the whole Starbucks account was fictitious.
Another candidate (again, from a reputed company with 11 years of experience) said HTML was dead because Visual Studio was available. I have no idea what he meant by that - and was afraid to ask.

Most senior people (tech leads and architects) I meet from the Indian market have no clue about the whole wide world of public website development, its special set of concerns and even about the niche world of Rich Internet Applications. I can actually find college kids who know more about social apps, widgets, Open Social and the mobile web. This is a matter of concern because some of the coolest, cutting edge technological developments in the past few years have been in that domain.
For the offshore services industry, we need more architects and leads who have worked with a set of technologies that I list below. If you are out there and have a good idea about these, give us a shout out.

1. Most clients who come to us start with the traditional web requirements - enhance the search engine ratings for their website. This is the area of Search Engine Optimization. From a technical perspective, it requires the knowledge of web standards; the optimal usage of Javascript and CSS; Cascading Stylesheets 2.0; Keyword analysis; the application of analytics to figure out usage patterns.
Clients also request us to build their sites to be compatible with various browsers; and be XHTML compliant.
Sometimes they ask us to analyze the site from a usability perspective - do users prefer a certain color over another? Do the right hand navigational links work? This is the area of usability engineering tied in with accessibility for the disabled.
(As a side note web accessibility guidelines are mostly ignored by website builders. Soon, such guidelines may be mandatory. Read this link for more information on accessibility)
Clients ask for the addition of rich internet features such as partial page refreshes or drag and drop. These require an architectural knowledge of Ajax and also the several Javascript libraries of effects (starting from prototype and scriptaculous to Dojo).
Optimization of the browser rendering time is a priority for many clients. This requires knowledge ranging from the HTTP protocol to DOM traversals in Firefox.
That is just the range for traditional requirements.

2. Clients then transition to what are called "Web 2.0" features - primarily User Generated Content (UGC) and User Generated Categorization (taxonomies).
Clients at this point need help on prominent Content Management Systems (for web content management). They need the ease to integrate different "Content Types" such as blogs, wikis, photo albums and yes, video sharing to their websites. They want tagging support. They want to easily add a comments module or a rating module or a polling module to a blog or photo album. Some clients even want video editing features added to their websites.

3. The marketing guys then kick in with the need for Social Networking to be added on to each website. This is such a huge topic in itself that I will write more later. The design for a social networking schema is not trivial. Add 100000 users and you could have a mess in your hands.
Since Facebook started allowing third party applications in their profile pages, there has been a flood of initiatives to build a "widget" framework. One of the most popular is "OpenSocial". OpenSocial is so cutting edge that the open source community is still working on a Open Social Container. Go, check it out.

4. Then the website goes Mobile - the request that a website display a good usable version on all the mobile devices around the world. There are so many models and so many standards around this. To design an appropriate solution is a challenge.

As you can see, there is no dearth of challenges here. Yet, I do not understand why our senior technical personnel won't step beyond Object Relational Mapping or Service Oriented Architecture. Let alone step beyond, why do they not even have an awareness about the challenges of public websites?

The two trends that I have seen of these domains is that
a) more work is done by the open source community and startups and

b)if you have been USING websites from these domains, it helps you give a headstart.

The second point is probably why college kids are better informed about social networking, mobile applications or blogging - because they actually use Orkut or MySpace; and harass their family for newer mobile models with better online features.

Meanwhile, I am going to keep looking for an architect who understands the term "Social Graph".

To be known as an expert

2008-06-02T10:14:00.000-07:00

I have been working in the IT services industry for the last decade. You could say I have come up the ladder a few rungs. Time for some introspection.
My job leads me to interact with clients closely.
In our business there is a need to impress clients. Clients have to believe they are interacting with an expert and competent person. Ideally, for a person like me, that means that I should be an expert in my subject matter (instead of making them believe that I am an expert).

There are several problems with being an expert - or, more precisely, acheiving expertise.

The Major issue is this: My job may not REQUIRE me to be an expert. Typically all that an organization needs is an illusion of expertise. It is possible for a moderately smart person to communicate well and project an aura of expertise. It is strictly possible that this is what a services vendor requires.
In fact, this is what most medium-sized services companies can aim for. Given a new subject matter (such as Social networking) that you are trying to gain access to, it is near impossible to find a person sitting in the wings to take control of the project. So, you end up having a pool of talented talkers who can project expertise. Over time, after you have completed a couple of projects in a domain, you can hope for some real expertise - if the person has not been moved on to something else.
The other option is for a services company to cater to a niche market such as Rich Internet Applications (RIA). If you did, you are close to what is happening "out there" and you can truly build on past expertise.
Of these two options, most outsourced vendors would go for the first option - in which they offer a broad range of services in different domains (and make it up as they go).
So, if you work for the normal vendor doing business in India, your job may require you to be a fast talker with little time (or need) to develop true expertise.

So, sometimes, I stand upon the threshold of a new and exciting technology and wonder - is it worth it to delve deep into it?

The fantastic vision of a true expert is very attractive - but not well defined. If I am a Sharepoint expert, by my own standards, I need to be able to do the range from:
- creating a workflow for approving and publishing infopath forms
- setting up enterprise search edition in a production environment with optimized indexing and storage.

Thus, expertise consists of conceptual ideas of architecture; it consists of a knowledge of practical thumb rules and quirks in implementation; but more than anything, it consists of actual experience.
The problem is an incomplete conceptual idea and a set of practical guidance CAN be obtained from Googling a subject or reading up a manual. Too often, that is mistaken for expertise. In fact, too often, that is passed off as expertise.
I do not think a person can claim to be an expert in a certain technological area unless they have actually gone through three or four different implementations and projects. Experience Matters!

One year back, I was walking along a corridor at work, when a senior manager called me aside and said that some developers needed assistance with a product called Drupal. I had heard of Drupal but that was it. But I was in a comfortable situation because the clients would never be exposed to me. I just had to work in the background.
I installed Drupal that day and over a period of two weeks could create two modules using PHP (a platform then unknown to me). In the process I learnt about MySql and InnoDb and Apache Virtual Hosts and Drupal hooks. I worked 16 hours a day and enjoyed every minute of it.
Soon, I found that other projects were using Drupal and tried to get together for best practices. I could perform Drupal code reviews and in general learn the way of Open Source in a few short months.
Am I an expert in Drupal? I don't think so - I could see myself becoming one if I worked on more projects and deployments. But the key factor that gained me entry was the fact that I was isolated from the clients during the initial period.

Now, let me highlight the problem - if you are a senior technologist in a services vendor, you WILL be put in front of clients as an expert. That is the best approach for the vendor in a scarce talent pool. But that is of course the perfectly wrong way to gain expertise - sitting with clients under pressure to prove yourself. This is the basic problem for concerned technologists.

I have thought about this and understood that the service vendors have little choice in this matter. They face stiff competition and everyone out there is doing the same thing. We can either have the business or lose it.
Also, mostly, the basic definition of expertise differs between the sales person or business manager and the technologist.
What do we do?

Drupal - Common Do's, Don'ts

2008-03-13T06:45:00.000-07:00

If you are starting up a Drupal project or you are analyzing one, here are a few Do's and Don'ts:
1. When you are developing with Drupal you will be required to create custom modules or extend existing modules. Please remember that Drupal is a developing product and therefore to manage upgrades they recommend a few standards. The first one is that any custom modules you develop should NOT be under the modules folder. Create them under the /sites/all folder. You can still install them and configure them from the administrative interface. This way custom modules are isolated from Drupal core modules.

2. It is tempting in Open Source frameworks to modify the core code. This is generally not necessary in Drupal. As an example, to create a custom block in a profile page with user information, you need not modify the User module. You can add a file called template.php in your theme's folder. You can then override the theme_user_profile method, by adding a _user_profile method.

3. Similarly, it is a common requirement to select a subset of content at some point. Let us say your content depends on an additional key such as department. So for a particular department you want to show content only for that department. You should NOT modify the node module to accomplish this. The node module and other core Drupal modules allow you to modify the queries they send to the database by using the db_rewrite_sql hook. You can add your own custom filters (such as the department id) to a query created by the node module by overriding that hook.

4. If you do modify core Drupal modules, keep track of which were modified and where, by using commenting patterns.

5. Use the Filtered HTML filter always in user content so that you can protect the site from Cross-site Scripting attacks.

6. Use the .install files in your custom modules to install custom tables in the database. Avoid creating your own MySql (or Postgres) scripts to run separately.

7. If you are using Ajax in a Drupal site use the JQuery functions instead of rolling your own. JQuery is packaged with Drupal and it is easier handling Ajax through it.

The key idea is that when you think of modifying core Drupal code, most of the time you can get away by extending existing modules, or adding hooks. The framework is built with the idea that it should be extensible, not mutable. If you spend some time figuring out the way to extend a module (instead of hacking into the core modules) you will avoid lots of headache later.

Lessons from a Live Server Issue in .NET

2008-03-13T01:47:00.000-07:00

(Updated Below)
A couple of months back my company was asked to upgrade the MCMS based public site of a client. We performed the upgrade for a subsite from .NET 1.1 to .NET 2.0. When we rolled out the upgraded code, we also changed the .NET runtime in their main rootsite and other virtual directories to .NET 2.0 runtime.
The load balanced site ran for 4 hours on both servers and then started giving "server very busy" (HTTP 503) errors. We were alerted and then worked off and on for 3 weeks and found the problem.
The problem had to do with HttpWebRequest-Response objects in .NET 1.1 and 2.0.
The 1.1 and 2.0 documentation for HttpWebResponse clearly states that not closing the response stream will cause a server to run out of connections. The behavior thereafter is not defined, but we found the behavior alright - subsequent threads that try to open the response stream just HANG. They block the entire request execution and the server starts throwing HTTP 503 errors.

Here is the offending code:

string sURL = "http://localhost/NR/...";
HttpWebRequest wReq = (HttpWebRequest)WebRequest.Create(sURL);
wReq.Method = "HEAD";
HttpWebResponse resp = (HttpWebResponse)(wReq).GetResponse();
string cntLen = resp.Headers["Content-length"];

This code does NOT close the response object. It should call “resp.Close();”.

Yet, this code worked fine in the client's live environment for 2 years. Why?
We found that .NET 1.1 actually recovers the response stream even though the documentation says otherwise. That is, there is a loophole in .NET 1.1 for people who do not close the response stream.
In .NET 2.0, that loophole is closed. Therefore, the above codebase hangs the server if upgraded to .NET 2.0.

Lessons we learnt
This was a contentious engagement with accusations and defenses about what went wrong. But we learnt a few lessons that I am documenting here:
1. When we upgrade or make a change in an environment and the server starts misbehaving ALWAYS suspect the upgrade or the change. With automatic windows updating and external system dependencies, it is easy to dismiss a problem as coincidental to an upgrade. Most of the time that is not the case. Most of the time it is a change that was consciously made.
In our case, the code that we actually upgraded to 2.0 was working fine. It was the legacy 1.1 code that was running in the 2.0 runtime that showed issues. Therefore, from the first, we thought the upgrade was NOT the cause. Of course, it turned out it was.

2. Always suspect code. If threads hang or we get weird errors in an environment with load, it is natural to suspect that the code is not the problem. We tend to suspect external system configurations, any change to configuration, or hardware issues. In my experience, things keep coming back to the code. Suspect the code first and do not be afraid of debugging in the server with trace statements, file logs - anything.

3. Always check connections to external systems. More specifically, suspect resources such as database connection pools, socket connections, web services. The above point (point 2) should be used in conjunction with this - instead of randomly checking the code or code reviews, check wherever external connections such as sockets or database readers are opened.

4. One thing that helps immensely is a simple stress test with a tool such as Web Application Stress Tool (WAST) from Microsoft. It is free and there is no reason not to record a session and run a test with 50 threads for 30 minutes every time you make major changes. I also suggest running such a test in every environment - not just staging or live. Run it in your own computer, if possible.

5. Do not hesitate to put trace statements or Response.Writes or writes to a file in code in the server - if time is of essence. This is an advantage with ASP.NET 2.0's dynamic compilation or even legacy ASP and PHP. I found the threads hanging by using System.IO.File.WriteAllText method. It takes a path+filename and a string and writes the string to that file. By putting debug statements in different parts of the code, I could find the last place where the code was stuck.

Update
The performance counters that were most useful during this issue were the ASP.NET 2.0 (or ASP.NET 1.1) object with the counters Requests Queued and Requests Current.
During stress testing the criteria that was most useful in WAST was Socket Timeouts. There were many timeouts and after fixing the issue there were none.

Developing Facebook Widgets

2008-01-19T21:17:00.000-08:00

Facebook.com is one of the earliest social networking sites that opened up for third party applications. In this tract, I will try to explain how a basic Facebook application works.

What is Social Networking

Facebook is a social networking website (among many such as MySpace, Orkut) - it allows users to be part of a community. They can communicate with each other through a profile page. A user's profile is typically public, or exposed only to friends or completely private. Users can share videos, photos and other information through the community.
This simple concept (in an alternative view) means users have an online "persona" - an anonymous personality where they can network with other people around the world with like minds. Facebook's founder calls this a "social graph".
There are specific social networking sites that allow business networking - LinkedIn is one such site. Being a member in Linked In, you can network with other people in the same business and even get references from your coworkers.

Why Facebook Applications are important
Facebook was one of the first social networking sites that allowed third party applications to be exposed through Facebook. This is a very strategic move. What do we mean by third party applications?
Let us take a chatting application. In a social networking site, live chat is not typically inbuilt. Users have to use their own Instant Messaging clients. So, it is a service that facebook users would love to have.
Now, Company A is creating a chat application by itself. It is trying to attract web users to use its IM.
Facebook brings Company A and facebook users together by allowing Company A to expose its chat application through facebook.
Company A creates a facebook "application" or "widget". This application follows facebook's standards and will work well within facebook. (We will cover details of this later).
Users can add a facebook application to their profile. Now when a bunch of users add the chat application, they can send invites to their friends to add it also. Thus, if an application is good, it can spread very fast - this is called "Viral Marketing" or the "Viral Spread". Such users may very well be interested in downloading the independent IM client.
Thus Company A has very quickly attracted loyal followers to itself with a simple widget.
The facebook widget model has its limitations but it has lots of potential. Facebook itself has a few default widgets, such as Photo sharing and Events sharing. You could very easily search for other widgets; check their popularity; read reviews about them; and then add them to your profile.
For example there is a very popular widget for film reviews called Flixster in Facebook. It allows you to share movie reviews and also rate movies. Blockbuster, the popular video rental chain in the United States has its own facebook application.
(I use the terms facebook application and widget interchangeably - facebook calls them applications but the more standard industry term is widget)

The broader meaning of Facebook applications
Although facebook applications are very popular and useful, they cannot really be shared with other websites. Google is coming up with a widget standard called "Open Social" that will help different vendors to share widgets.
The race is on between facebook style widgets and OpenSocial style widgets.
This enables and revolutionizes social networking and web marketing - it fulfills the promise that different social networking siets can talk to each other and widgets will be portable across sites. That implies that users will finally have a single addressable online presence and be able to conduct all their online interactions through such networks and widgets. It means a different development model for vendors.

The anatomy of a facebook widget
The rest of this article details how you could implement a simple facebook widget.
Let us say Company A wants to develop a widget for Movie rating. They would like users to share their reviews with each other and rate such movies. What would Company A have to do to develop such a facebook widget?
First Company A needs a web server that hosts their widget web services and a database server that hosts their application database. The webserver can be Apache or IIS or any other stack. Company A can pretty much develop the applications in any server side language - .NET, Java, Perl, PHP, Python or Ruby. It does not matter. And of course, any database server.
Let us say Company A is hosted in www.companya.com.
In the development of this application, Company A needs to show information to users in their profile page. If a user adds this application,
- they will see the widget's view in the profile page
- Movie rating or reviews can be done from the widget's "canvas" page.
The programming needs for the profile page view and the canvas page view are different.

The first few steps to facebook development are:
a. Add the Facebook Developer Application to your facebook profile. This helps you register and obtain the keys.
(See: http://developers.facebook.com/get_started.php)

b. Register your application with facebook by supplying a callback url (ex - www.companyA.com/movies/process.php) and a brief description of the application so that users can search for it and add the application.
b. Download the facebook api wrapper in your language of choice. This also comes with a sample application.
c. Create an application key and a "secret key" from facebook - the api needs this for recognizing your application.

Take a look at this page for the facebook API documentation and the anatomy of a facebook application.

1. The Canvas Page View
In the canvas page, the user will see a list of movies that have recent reviews. the user can add comments to the reviews. They can enter thir own reviews using (say) a rich text editor.
When a user views the canvas page (they reach it by clicking on the application name in their list of applications), facebook actually internally calls your registered callback url (www.companyA.com). Your web server recieves the request and you are free to serve any page view you want. (That is, facebook proxies for you). Facebook, while rendering your returned HTML, parses and manipulated the Javascript, links and other DOM elements in the page. Other than that, the page is displayed with your themes. It is rendered within the facebook window.
(The other alternative to such "proxying" is, of course, to use iframes in the page. This is the approach that Open Social takes).

Thus the canvas page view is synchronous - facebook actually calls your url and renders back the page returned by you.
When facebook calls you, you can save the review entered by the user in your database. Facebook supplies the facebook user id of the user. You can retrieve additional information from facebook itself by calling facebook's webservice API. We will later see an example of where you will need this.

2. The Profile Page View
The profile page of the user is unique and it has certain safety requirements - facebook knows that the profile page of the user is shared by many different applications. Therefore facebook does NOT call your server when it renders the profile page.
In our movie rating example, when a user views their profile page, let us say we want to show them a set of comments on their reviews. The problem is facebook does not call companyA.com when it renders the profile page. Therefore, we have to "deposit" the profile page view of an user when anyone enters comments on their review. We have to call the facebook API call set_profile_FBML and pass in the markup to show.
Facebook takes that deposited markup (called FBML because it has a slightly different set of tags than HTML) and displays it to the user when they hit their profile page.
To show accurate, on time information 9such as stock prices) in the profile page you have to set up a different kind of call back.

Thus, you can see that the interaction involves going back and forth between your web server and facebook.
In the canvas page, face book calls your server and you may in the background, call the facebook API back and then return information to the user. (Calls to the facebook API can be slow).
In the profile page, facebook takes the deposited FBML from its own database and shows it to the user. While processing that FBML, it may call back your web server.

For longer running interactions, it is also possible to run background services that can call the facebook api using an "infinite session". By default, facebook API is called under a user's credentials (context).
Storing certain types of information from facebook API (such as an user's friend list) is illegal.
Facebook's API has frequent changes - be on the lookout for breaking changes by monitoring their newsletter.

For more information:
- There is a facebook developer toolkit in .NET in Codeplex - here.
- The facebook wiki and API documentation is here
- For information on Open Social please see here.

Does Web 2.0 exist or is it a buzzword?

2007-12-09T22:53:00.000-08:00

Whenever I discuss Web 2.0 with a group of developers they are confused about what it means. I have referred to O’Reilly’s definition, but it is way too generic. I have pointed out that there is a Web 2.0 style in Graphic Design and that does not interest them. Every service vendor out there is talking about Web 2.0 and job portals have Web 2.0 specialists.
In spite of all this hype, it is still hard to explain what web 2.0 really means. One friend suggested content sharing between users; another mentioned Ajax based websites. There are still lots of technical people who are skeptical that there IS a "Web 2.0".

I have realized that the core fault is in asking if there is a web 2.0 to the wrong group of people – it is pointless asking developers or service providers whether they can code web 2.0 or they offer web 2.0. The true set of people to ask about web 2.0 are web users – not the builders.

Internet users have been using the web since 95. Many more begin to use it every year. These are the best people to ask – do you see a way in which your usage of the web has become different? Do you see a better web to call it web 2.0? We should be asking the political blog writer who writes commentary. We should ask the student who uses the web to research and collect information. We should even ask developers whether they USE the web differently.

Forget about Semantic Markup, W3C compliance or cross browser – what do the millions of web users see different?
As such an user myself, avid follower of political blogs and blog writer, I see such a change myself. (That change has nothing to do with Ajax).

The primary way in which the web has changed for me is in simple services that I am able to avail of and then link with other people around the world very easily. The ability to form a kind of community fast is the primary reason I feel the the web has changed.

One example is del.icio.us. By storing links in del.icio.us, I can actually observe that other people stored the same links and so take a jump in to see if they have more interesting information and so on in a rapidly complex linking mechanism. These links that del.icio.us forms sits one layer above the hyper text linking at the core of web 1.0 itself.
Reddit.com offers similar services – the ability to discuss within a community about an article.
Then of course RSS – that I can subscribe to feeds and pull information from the internet for my own content construction is amazing to me.
If I am a content author, if I want to take my content around to a community (flickr, youtube, blogger), , if I want to establish a community of like-minded people (Facebook, Orkut, MySpace), if I want to have the ability to combine this content with others effectively (Mashups and RSS), then the current web is way different from the original web.
Do not tell me that these facilities existed in some form in the original web - I don't care. These are much more available, visible and usable now and that is what I care about.

These features have nothing to do with the way the websites are built technically. They have to do with adding personal and social value.

I think Wikipedia and similar ideas are the greatest achievements in human collaboration – because they fulfill the original goal of the internet. The original goal was that all human knowledge, historical and otherwise should be available in an easily accessible, hyper-linked form. Web 2.0 adds the ability to form communities on top of this massive data and analyze it and parse it and add additional information.

Just for that, Web 2.0 does exist – it is not a phantom buzz word. It is a real phenomenon and we have to be proud of it.

The code of IT consulting

2007-10-21T21:33:00.000-07:00

I have been in the IT consulting services industry for some time now. I thought reflecting back on certain practices as a consultant will be useful:
Early on, when I worked in Cognizant, I was new to the contracting model. The idea in contracting is that a temporary worker performs certain services on a project. This helps the client to avoid directly hiring, say a programmer. In the United States, hiring an employee comes with significant overhead – such as providing health insurance and other social security benefits. It is also not easy to let an employee go without proper cause.
The client taps into the contractor pool and hires a consultant. The client also has another advantage – they can hire a specialist specific to the current project. They do not have to invest in training.
Thus a client hires a consultant for two reasons:
1. They can make targeted hirings for certain projects that are immediately available.
2. They can get a specialist in a particular field.

If you are an Indian going to the United States for working in the IT field, you have two options:
1. Become a permanent employee in a company
2. Work as a consultant
As a permanent employee, you gain job security – as mentioned above, you usually get social security and health benefits. The company will also think twice before laying you off. But you do not usually get paid overtime.
As a consultant, you work as an independent consultant or as the part of a consulting group. You can command hourly rates that result in a higher pay monthly than your permanent counterpart. But it is likely you have to pay for your own health insurance. You may lose your job any time.
When the economy is doing good, consultants make a killing because IT budgets are higher. When the economy is going bad, the first people to be hit are consultants (first-hand experience).
When a client selects you, they usually go through a primary vendor who makes consultants available to them. The primary vendor properly screens (that is, interviews) the potential consultant.

So, this is the situation in the United States. What about India?
Indian IT industry generally does not use consultants – for a couple of reasons. Because of the higher labor pool and the scarcity of jobs, Indian companies can easily hire qualified people for lower pay. Also, Indian IT industry does NOT have any problems laying off people.
In India Graphic Designers usually work freelance and as contract workers – but programmers rarely do.

But, when you get hired by an Indian company, you are still a consultant for the client (usually American or European). Your company (whether it is Infosys or Wipro or Satyam) basically uses you as a “body” to throw in and out of a project (hence the term “bodyshopper”). The large majority of technical people are bodyshopped.
There are two different models in Indian IT services:
Turn-key, fixed-bid projects – Projects that companies undertake to finish with available resources - since they have specialists in the field. This could be an SAP implementation or a Business Intelligence application. The clients in this case rarely have visibility or choice in the allocation of individual resources. The projects can be long running projects (as in SAP) or shorter projects meant for an “evaluation”.
Retainer projects – In such projects, the client uses the offshore consultants as an extension of their own development team. In such projects, the resources can be used in an hourly rate and are called “Time & Material” resources.
In either case, the key is to understand that you are primarily a consultant – you are being paid for rendering services to the client. You are being paid to work in the best interests of the client.
In my mind, this is the best model to work under – one of my former colleagues thought ALL IT workers should be consultants. I have noticed that in the companies I worked for, the best work was done by consultants; and they were the most knowledgeable and responsive. One reason is that the SLA (Service Level Agreement) is clear in the case of a consultant – that you respond to client emails fast is clear. To an employee, somehow the SLA is not clear.
In my work, I try to act as a consultant (ie as if there is an SLA) to everyone – even fellow employees; and I try to honor such a code when I respond to calls from other divisions.

What does such a “code for consultants” mean? You will see as you read below that most of it applies to other divisions in your own company. Treat everyone as a client at work – including your subordinates.

The code of consultanta
1. Adher to an email etiquette – always respond to any client emails within a day; you can tell them you are working on something else. But, respond!

2. Always make sure the clients can reach you through different channels – a phone, if they cannot reach you online.

3. Make sure they have an escalation path – so that they can reach your boss or someone else who can help them. I have seen consultants enforce their indispensability by hiding other people in their organization. That is good for you but not for the client (which ultimately will affect you). In fact I would suggest making sure they know your whole team. The escalation path should be available even if you are not on vacation.

4. When you go on vacation, make sure you have an auto-reply set that says when you will be back and who the client can contact in that time.

5. Make sure that all communications are polite – and be secure enough to back off when the client want it their way.

6. Keep an eye open for any collaboration or value-add for the client. If you see one of your other divisions create a training plan, see if it is applicable to your client. Alert them about new information in the market. Make them understand you have their best interests in mind.

7. Always keep track of your time to as much granularity as possible – if a client loses confidence in your timesheets, the relationship is over.

8. I have noticed that Indian teams become very upset when the client hires people on their own. (God forbid if the hire is an Indian). I have actually seen offshore managers try to strategize around this. The offshore team feels paranoid; and the managers feed this paranoia. I have seen team members trying to eavesdrop on conversations! This is a very bad and insecure way to deal with such hirings. As consultants, it is our responsibility to SUPPORT their hires in every way. I have always tried to do so, and I have not had reason to regret it. Instruct your team not to be paranoid and extend all cooperation. Do not sabotage or second guess the client’s actions and do not let your team do so either.

9. Never, never solicit employment from a client when you are working as a consultant. If the clients themself bring it up, tell them to contact your company. Usually consulting services have a Non-Solicit and a Non-Compete agreement with you to prevent this. Even if your company does not have such agreements, never solicit employment from a client. It is unethical.

10. If someone asks you who you work for, mention your company name – NOT your client’s name. I have seen people do this in interviews – if their client is Dell or Intel, they say they work for Dell or Intel. They don’t – they work for the company that processes their paycheck.

Technical Interview Preparation - Essential Skills

2007-09-09T03:09:00.000-07:00

Over the years I have interviewed hundreds of candidates for technical positions as well as project management. It is frustrating to see how ill-prepared candidates are for such interviews. For clearing a technical interview for a developer or a senior developer, it is important that a candidate be knowledgeable in a few subjects.
I will first list what are essential skills and then we will cover what makes you an expert. The focus mostly is on Microsoft technologies.

Essential Skills
1. HTML - it is a must, to know HTML. It is pointless in this day and age to say you only know desktop apps.
2. Javascript - atleast to the extent of the standard way of getting the value of a text box; to handle a click event.
3. Regular Expressions - most people do not know regular expressions. Try to find out how to validate strings using regular expressions in Javascript, and a server side language such as Java, C# or VB.NET.
4. XML, XPath, Parsers - Know something about well-formed XML; about parsing XML documents; and querying XML documents using XPath. Try using a client side parser and a server side parser.
5. XSLT - is good to know.
6. Object Oriented Programming (OOP) - Polymorphism, Inheritance, Encapsulation, Overloading, Interfaces, Abstract Classes. You should be able to create a simple class model.
7. Collections - Arrays, Hashtables, Vectors, Dictionaries; whatever they are called, learn how to use a list of objects of same type or different types and how to use them in any language of your choice.
8. Database Access - in any framework of your choice; .NET, Java or PHP.
9. SQL - It is essential to know how you can query a set of related database tables.
10. Table Design - for a typical scenario such as online shopping or student registration or an address book. You need to be able to design tables and relate them using keys.
11. Tools - such as source control.
12. Deployment - you need to understand how the servers are laid out in a production system. You need to know how your development code is transferred to QA.
12. Concepts such as what a web server is; what a database server is; what is QA; the role of a client and a server and so on.

If you knew all of the above, a company will consider recruiting you and training you. You can learn most of these with Open Source tools.
Of course, for real expertise, you need to know a lot more in web applications and desktop applications. I will list that out in subsequent posts.

The role of project managers

2007-09-02T19:45:00.000-07:00

What is the role of a project manager in a software development consultancy?
I have worked in different structures in India and the USA. In India, in Cognizant and other companies, technical staff became Project Managers (PM). They guide the project from collecting requirements to delivery. But they also technically mentored developers; they were in charge of technical architecture. They conducted the appraisals of technical staff.
In the USA I was exposed to a different structure (that my company called the Matrix structure). Developers reported to a tech lead and then a set of architects and so on upto the CTO. The PMs were part of a completely different reporting hierarchy. They actually went to a different VP. On projects, the PMs and Tech Leads work together. PMs have no say in technical decisions. They are in fact, completely non-technical. Appraisals and mentoring are done by the Tech Leads.
I, as a technical person, felt comfortable working in the second structure. But I need to mention that the second structure was not for a software consulting company.

When I came back to India and joined my current company, I noticed that the structure was similar to the American model. I fit in fine.
But I do see that there are some confusions about what a project manager's role is with respect to developers. I am trying to gain some clarity on that.

In the Software Development methodologies such as Microsoft Solution Framework(MSF) or Rational Unified Process (RUP), the project manager has an important role. That role is to handle requirements, change requests and risk. We should note that developers ALSO have a very important role. In the eyes of these processes, every role has a value. As such all the roles are in the same level. I want to reemphasize this - in a software project, the PM has a role at the same level as a developer or a Tech Lead.
As a part of their role, they have certain responsibilities.

In consulting organizations, PM is ALSO a Position. You enter an organization as a PM. That has commensurate pay and other perks. They also have certain administrative responsibilities.
Thus the PM has a position as well as a role. In a project, the PM has to play their role. The division of labor between the Tech Lead, developers and the PM is clear. As long as we stick to that we should be fine.
But, I have seen PMs forgetting this and get antsy with developers. I have seen this so often with Indian PMs that it is annoying. For example, PMs try to second guess the Tech Lead's estimates. I had one PM sending an email questioning a row in a database table. PMs also tend to take MORE ownership than necessary in a project lifecycle. Some of them yell and scream and write nasty emails to the technical staff. This commonly leads to unhappy developers.
As part of their role, PMs have to be in constant consultation with their technical staff - particularly with Tech Leads. It is important to have a good relationship with tech leads.
We have to keep the technical staff happy - that is a priority in any organization that depends on software development.

I have talked about this with some PMs and their response is that they are answerable to clients. This is such a cliche - I have never heard of any PM being fired because a technical mishap happened in a project. Developers deal with very uncertain technology; and they have to stick to very tight timelines. That is a given and MOST clients understand that.
Obviously there is another side of the coin - there IS unprofessional behavior among technical staff as well. But that can be combated using standard warnings and escalations.

Interactions with American clients - Twenty tips

2007-07-27T22:31:00.000-07:00

Dealing with any clients in services is an art. In the case of IT, we primarily deal with American clients (atleast in Photon). It is useful to know how the English language works with them.

1. Do not write "the same" in an email - it makes little sense to them.
Example -

I will try to organize the project artifacts and inform you of the same when it is done

This is somewhat an Indian construct. It is better written simply as:

I will try to organize the project artifacts and inform you when that is done

2. Do not write or say, "I have some doubts on this issue"
The term "Doubt" is used in the sense of doubting someone - we use this term because in Indian languages (such as Tamil), the word for a "doubt" and a "question" is the same.
The correct usage (for clients) is:

I have a few questions on this issue

3. The term "regard" is not used much in American English. They usually do not say "regarding this issue" or "with regard to this".
Simply use, "about this issue".

4. Do not say "Pardon" when you want someone to repeat what they said. The word "Pardon" is unusual for them and is somewhat formal.

5. Americans do not understand most of the Indian accent immediately - I have heard some of them say that they only understand 75% of what we speak and then interpret the rest. Therefore try not to use shortcut terms such as "Can't" or "Don't". Use the expanded "Cannot" or "Do not".

6. Do not use the term "screwed up" liberally. If a situation is not good, it is better to say, "The situation is messed up". Do not use words such as "shucks", or "pissed off".

7. As a general matter of form, Indians interrupt each other constantly in meetings - do NOT interrupt a client when they are speaking. Over the phone, there could be delays - but wait for a short time before responding.

8. When explaining some complex issue, stop occasionally and ask "Does that make sense?". This is preferrable than "Do you understand me?"

9. In email communications, use proper punctuation. To explain something, without breaking your flow, use semicolons, hyphens or paranthesis. As an example:

You have entered a new bug (the popup not showing up) in the defect tracking system; we could not reproduce it - although, a screenshot would help.

Notice that a reference to the actual bug is added in paranthesis so that the sentence flow is not broken. Break a long sentence using such punctuation.

10. In American English, a mail is a posted letter. An email is electronic mail. When you say

"I mailed the information to you"

, it means you sent an actual letter or package through the postal system.
The correct usage is:

"I emailed the information to you"

11. To "prepone" an appointment is an Indian usage. There is no actual word called prepone. You can "advance" an appointment.

12. In the term "N-tier Architecture" or "3-tier Architecture", the word "tier" is NOT pronounced as "Tire". I have seen many people pronounce it this way. The correct pronunciation is "tea-yar". The "ti" is pronounced as "tea".

13. The usages "September End", "Month End", "Day End" are not understood well by Americans. They use these as "End of September", "End of Month" or "End of Day".

14. Americans have weird conventions for time - when they say the time is "Quarter Of One", they mean the time is 1:15. Better to ask them the exact time.

15. Indians commonly use the terms "Today Evening", "Today Night". These are not correct; "Today" means "This Day" where the Day stands for Daytime. Therefore "Today Night" is confusing. The correct usages are: "This Evening", "Tonight".
That applies for "Yesterday Night" and "Yesterday Evening". The correct usages are: "Last Night" and "Last Evening".

16. When Americans want to know the time, it is usual for them to say, "Do you have the time?". Which makes no sense to an indian.

17. There is no word called "Updation". You update somebody. You wait for updates to happen to the database. Avoid saying "Updation".

18. When you talk with someone for the first time, refer to them as they refer to you - in America, the first conversation usually starts by using the first name. Therefore you can use the first name of a client. Do not say "Sir". Do not call women "Madam".

19. It is usual convention in initial emails (particularly technical) to expand abbreviations, this way:

We are planning to use the Java API For Registry (JAXR).

After mentioning the expanded form once, subsequently you can use the abbreviation.

20. Make sure you always have a subject in your emails and that the subject is relevant. Do not use a subject line such as "Hi".

Update
Ok, One of my friends forwarded me an email a couple of days back - in which someone STOLE this blog post; added some formatting; and are now sending it as a forward to everyone.
This is my first exposure to IP theft - and whoever took it should have atleast attributed to me.
Now, they have also added four other tips - and these are completely wrong. Let me list them here:
21. Avoid using "Back" instead of "Back" Use "ago".Back is the worst word for American.(for Days use "Ago",For hours use "before")

[Ram]You CAN say "Back". In fact Americans understand "back" better than "ago". Nobody says "four days ago" in the USA.

22.Avoid using "but" instead of "But" Use "However".

[Ram] It depends on the context. You CAN say "But".

23.Avoid using "Yesterday" hereafter use "Last day".

[Ram]Whoever says "I had dinner Last Day?". This is idiotic. You CAN say "Yesterday". It is the right thing to say.

24.Avoid using "Tomorrow" hereafter use "Next day".

[Ram]Again, whoever says "I will meet you Next Day?". You CAN say "Tomorrow".

Whoever took my post and added the last four tips is clueless.

Application Framework - Project Breakdown

2007-07-19T20:12:00.000-07:00

We have to break down the project into different modules. Since these are dependent on each other, it is always a challenge to manage this:
1. Start with an Events namespace - to wrap asynchronous communication. This may need some thread management. It may help break it down into a set of Services and Tasks. A single task will wrap a thread. By doing this, we will be able to publish exceptions asynchronusly.
2. Build an Exceptions namespace - with a base exception and some synchronous and asynchronous logging capabilities. The log will be a stub for now - it may just log to the file system.
3. Build a Caching base that will hold a set of records in a refreshable cache.
4. Build a Configuration namespace for managing name-value configurations and caching them.
5. Build a Lookup Manager that loads readonly data and caches them, in lookup objects.
6. Build a Security namespace with a principal for forms authentication.
7. Build a Connection String Configuration Manager
8. Build a Persistence namespace that manages saving and loading objects.

The Code generator can be built independently.
The Connection String configuration and the Configuration namespace can also be built independently.
The Exception logging part can aslo be independent.
The rest of it may see some intense coding.

Framework Development - Code generation

2007-07-19T10:18:00.000-07:00

Persistence frameworks, for rapid development, usually ride on generated code. I have worked with different tools, custom, commercial, that let you point at a database table, or an XML file and generate classes. They also generate stored procedures for CRUD operations.
We faced several decisions regarding such code generators - should we use Codesmith, or build our own? Should we integrate with Visual Studio? How do we make sure the spewed code is not modified, but is extended correctly? Do we use the table as the master schema or spew XML and then generate code from XML?

Integrate with Visual Studio
I think to have a code generator plug-in is very convenient and increases productivity remarkably. But it is also troublesome to install and to supply configuration. Plus it is overkill upfront. We will use an independent generator for now and then develop (or buy) a plug in separately later.

How to extend spewed code
I have seen code generators that use a XML, XSLT model to generate code; or use templated code (as Codesmith does); or just write out lines of code from a generator. In general, you spew code from a database table schema. Then you go ahead and write some business logic in the spewed class. But the spewed class may need to be respewed for additional modifications.
To manage this, some tools detect that the spewed class has changed and show a manual merging tool.
Ideally the class file should not be modified at all - but that is almost impossible to enforce. I have seen some good innovative solutions using partial classes (in .NET 2.0). You split the same class into a spewed, unchangeable class file and a changeable class file. You can add business logic to the changeable class file.
You can manage the same using a spewed, unchageable base class and a changeable sub class.
Visual Studio, of course, spews code all the time - when designing WinForms screens. In MFC using Visual C++, they would just request you not to change some lines.
I think actively trying to prevent developers from changing the spewed code is kind of overkill (atleast in internal applications). I think you can use a convention of storing spewed classes in a separate folder and then hoping that nobody changes them. On the other extreme, we can avoid spewing code at all; generate and compile the code dynamically during the build process using a custom build action. That makes sure nobody can modify these classes. (XML Serialization does something similar).
For now, I am tending towards a loose approach - we spew the code and hope everybody changes the subclasses (which are also spewed along with the base classes). If necessary we will tighten up later (by taking the person who changes the spewed code outside and shooting him).

XML is king?
One of the ideas is that we spew XML (with column names, datatypes) from the table schema and use that as the basis for generating code. This has its advantages - you can add some custom properties in the XML and regenerate code. What if you wanted a property to be readonly? You can add that as a custom attribute to the XML.
The core issue is that a table schema does not contain ALL the information that you can play around with in a C# class. As we saw, marking a property as readonly is an example. So, philosphically I like the XML file idea. But it is not practical to maintain a separate spewed XML file AND a spewed class file.
The problem is that people can mess up the XML and cause the table schema and XML to grow apart.

So, for now, no XML. We will be generating code from a generator; storing in separate folders; and then subclassing from these classes, if we want to extend them.

Application Framework Development

2007-07-19T04:59:00.000-07:00

We are discussing the development of a Microsoft-Application-Block style framework that would offer the following services:
1. Persistence
2. Caching
3. Lookup Management
4. Workflow
5. Multi-threaded tasks
6. Code Generation (for sprocs, persistence classes, lookup classes)
7. Configuration Management for connection strings
8. Exception Management, Logging Tracing
9. Security (Authetication, Authorization)
10. Events and Asynchronous wrappers

The project is ambitious and we have a few models to choose from. I will be posting on this (hopefully) regularly from now on.
The purpose of this framework is to help quickly develop, intranet web applications - such as an internal Library Management App, a Resume Management App, a Timesheet Tracker.
We have not yet decided if the framework will be open source - we will cross that bridge when it comes. Let us see if we complete it first.

The first decision we made was that we would use .NET because all of us are familiar with it. But, we also decided we will keep the patterns documented so that we can port to Java at some point.
It will use .NET 2.0 and hence features such as Generics will be used. It will not use .NET 3.0 for Workflow Management - for now. I am leaving that option open. We will use patterns from the Workflow Foundation.

The applications that will be built on top of this framework can be web or windows based. I like service based applications - because they are easy to switch between using Ajax or Flex for front end. We do not plan to build our own UI controls.

C# is the language of our choice.
NUnit is our choice for a testing tool.

Our primary target database server will be SqlServer 2005. But, I want to design the framework with MySql support possible in the future. This is not very tough to do; the MS Data Access Application Block has a nice set of patterns for such database agnosticism.

It is always a challenge in these kind of frameworks to decide which areas should be delegated to third party providers (in open source). One example is the Logging framework - Log4net is an obvious choice. It supports logging to the file system, event log or to the database. It supports different levels of configuration based logging. The only issue I have with third party software, particularly open source, is that they do not upgrade to later versions at your convenience. I am almost convinced we should use Log4net. It is also an issue of complexity though. It is likely we will standardize on a database log. We know that database server will be Sql Server. Do we really need an "appender" based, extensible system such as Log4net provides?

Other than Log4net, the other area that may need a third party software (although not necessarily free) is Code Generation. I will write about this in detail in the next post.

We plan to follow Microsoft standard naming conventions.
Comments? What comments?

Reengineering a website

2007-07-04T15:26:00.000-07:00

I have been working on rearchitecting a public website for the last few months. The experience has been instructive.
The website has a large class library in VJ++ components; at least a dozen databases; and hundreds of ASP pages. Our goals were:
1. To shift the code base to .NET and use an application framework designed by us.
2. To exploit this opportunity to redesign the business logic; split it into a set of services; and thus simplify maintenance and enhance performance.

There were several major modules in the website. We decided to approach each of these separately and organize the project into several phases for each module. By doing so, we consciously tried to decouple these modules - if module A was dependent on module B, we would try to isolate these dependencies into a service layer that implemented a facade. For example, the member information was used by all modules - and so we built a read-only service that provided the other modules with the member information.

The approach we took for each major module was this:
Step 1 - Discovery
1. Collect a high number of internal documents - both business and technical - that were available and keep them in a repository.
2. Navigate through the website ourselves and try collecting Database profiler data (such as stored procedures executed) and fiddler data (fiddler is a http debugger from Microsoft). This gave us the client-to-middle tier interface and the middle tier-to-database interface. We put these together in a sequence diagram.
3. List the database tables and try to infer their role in the application flow. Tie these to the higher level workflow in the application.
4. If the existing middle-tier was well done, we will be lucky to find a set of manager or service interfaces that are used to encapsulate business logic. List such manager classes and their interfaces.
5. Talk to the business users. Walk through the backend administration systems, if any. usually big websites also have administrative systems (such as accounting billing systems) that write some data to the database tables. Taking a look at them helps us to understand the tables better and also keep those backoffice systems in mind when redesigning the main website.
At this stage the goal is to collect as much data as possible. Vague ideas about the future architecture of the system and the services boundary float around and are captured. But the goal is to get our heads around the system.
In itself, the Discovery phase is useful, because it lets us collect and organize informationin a single place. In our case we dumped this information into a huge document with many links to sub documents. It was raw data.

Step II - Analysis
In the second phase we block each different subsystems in the overall website and discuss their inner workings. The data we collected in Phase I is useful for this (particularly the Fiddler and Profiler information). We also organize the discovery documentation and enhance it with sequences and flow charts. This may need copying information from some existing documents.
The Analysis is best done as a team with at least 2 members. It is a good idea to include an existing resource working on the maintenance of the legacy website.
The output of analysis is a better organized document.
At this time, we can also document the rearchitectural ideas that we have; any model that we would like the new design to be built on. For example, we may think that a subsystem can be split into a read only service and a read-write service. We may also have concrete ideas on the database model. It is better to document these and start discussing merits and de-merits. Such discussions tend to vanish if not documented.

Step III - Building Specifications
For such reengineering projects, it is unusual to get concrete functional specifications (other than the single line "it should look and feel the same as the existing site").
Therefore, in this phase we can move to building an architecture document. This document will cover each one of the subsystems that we are redesigning. If there are new subsystems being introduced, it will cover that. It will cover the Rollout plan - this is critical in a website that cannot afford downtime. It will aslo cover any integration plans.
Some of the subsystems could be straight ports from J++ to .NET. These have to be mentioned in the architecture document.
Some of these redesigns may require:
Digging deeper into the existing code to uncover business logic or any violations of our model.
Building Proof-Of-Concept (POC) - in our case we had to do this for the Flash-to-service layer integration.

Step IV - Technical Specifications and Estimation
From this point on the project feels like a development project and goes through similar iterations.

The collection of as much data as possible about the existing (legacy) system is crucial for the rearchitecture to be a success.

Thanks to Mr.Jorge Gonzalez for sharing some of these approaches and ideas with me.

Rats like my books!

2007-04-19T11:02:00.000-07:00

A few days back, I was working in my home. It was midnight and the clock below struck twelve. I felt an eye on me...basically a "presence". You know that creepy feeling.
I turned suddenly to catch what I expected was a ghostly apparition floating around. Instead there was a small rat staring at me. The rat was not particularly upset at being caught redhanded. He just twitched his moustache as if to say, "Whazzup", and then turned leisurely and walked away. I am serious - I have always seen rats running around like crazy but this guy was a well-adjusted rat.
I yelled for my protector, my wife. She woke up; said she had seen the rat a couple of times walking around. I was mad at her for not killing him; she said if I was brave enough, I should do it myself.
Now, this was a taunt - the last time a rat visited my home, my solution was to lock the bathroom (because he was in it) and not go to the bathroom for an entire day till the maid came in.
I ended up treating her taunt as an authentication issue and tried to close all our windows every night. But this rat apparently (like the velociraptors in "Jurassic Park") was engineered by a mad scientist - he could open windows. I saw him continously take his walk at 12 midnight. I basically gave up trying to convince him to stay outside. We just started locking our room in, so that he could take his walk undisturbed.
Now what has this blog post got to do with technology?
I have a pretty good collection of technical books - books that I collected in the USA. I keep them all in a store room and keep loaning them out to the team in Photon. Many of these books do not seem to be returning. I was beginning to get concerned - I generally forgot which books were loaned out to whom.
Anyway, I walked in to that room to get a book and found a weird smell assaulting me. It had been some time and I was sure the smell was coming from the book collection.
I took a book out and found what made the rat take his walk.
He was "marking territory"; doing the same thing most Indian males do when they see a nice wall; to put it crudely, he was using my book collection as a very private, custom built designer toilet.
I was outraged by this travesty - I mean any self-respecting rat could have torn my books to shreds or other wise leave them with a shred of dignity. This rat, as my wife put it, was using it as a septic tank.
As I held my nose and stared at my poor books, an idea suddenly dawned in me - as Buddha said, this could be used an opportunity. I saw the sunny side of the rat menace.
I plan to email the staff tomorrow at work and point out that my books are "polluted". I am going to narrate the story of the rat.
I am confident that all the books will be nicely packaged and returned.

Are native languages "unprofessional"?

2007-03-31T09:56:00.000-07:00

Early on in Photon, we went to a party and the discussion came around to talking in Tamil at work.
One of the drunk developers felt passionately about this - he said in a meeting we should NEVER talk in Tamil. He said it was unprofessional.
I was surprised by the vehemence of this comment - I said if even one person who spoke in another language is present, this is true. But if a meeting consisted ONLY Tamil speakers, I do not see why it should be unprofessional to speak to each other in that language.
I also argued that if this was true, how far do you take this? Is it okay to speak Tamil in casual corridor meetings or is English mandatory for this? What if two people are talking business by the water cooler - can they talk in Tamil? Or does the English rule apply only when you walk into a conference room?
My point was that this rule was unnatural - and I was not objecting to it because I am a "fanatic" - but because it is contrary to natural behavior.
But, of course, the drunk guy veered completely away into how "even China" was switching to English. And how the whole reason India had succeeded in the IT industry was because of English.
This is a digression here -
China is not "switching" to English - Chinese are learning the English language as a language. They are not learning every subject out there in English - that is, there is no English medium in China; they are learning English as a second language.
This is vastly different from India where the middle and rich classes have lobbied for learning everything in English, taking things to such an extent that kids are actually being punished for talking to each other in their native language during recess. Some schools fine you for talking in the native language.
Whenever somebody raises this, he is branded a cheap fanatic or someone brainwashed by the politicians. I respect Tamil politicians for making this an issue.
Secondly, Indian success in the IT industry also happened because of many, many thousands of studnets who learnt in the native medium but learnt English as an additional language. By claiming English medium as a reason for Indian succcess in IT, I only see the usual attempts of Indian urbanized upper middle classes to corner credit for the IT boom.
Digression ends here
Anyway, that argument was fruitless because the guy ended up blaming Tamils for this "unique" behavior - and I had seen most Indian linguistic communities talk to each other in their language at work. By bringing race into the argument, he lost the argument.
This was an year back - I was thinking about this last week and asked another friend of mine this question - say you went to an interview in a company; the interviewer ascertains your native language; say you are Telugu. The interviewer is also Telugu and proceeds to interview in Telugu.
Would you join that company? Would you consider that unprofessional?
His answer was yes - he would consider it unprofessional and he would reconsider joining that company.
This friend of mine is far more reasonable so I could argue with him. My points were:
1. We use English as a common communication medium. That is the sole purpose. If there exists another common communication medium and if it is your native language, then doesn't it make sense to use that? It will probably relax both of you.
2. What makes English professional and Telugu unprofessional? What causes that judgement? Mind you, I am not talking about an artificiallly constructed "pure" Telugu conversation. If a normal conversation you have with a friend at work can be in Telugu, why can't it happen with a candidate?
I believe the one good reason to not ask the native language of a candidate is because it could result in discrimination. But then HR should not ask for age either (you should not discriminate based on age either). Most candidates will gladly mention even passport numbers in their resumes in India - so why is asking for and conversing in a native language such a bad idea?
I do think my friend is accurate about this - I think most people would react the same way. But I am not able to understand why such an opinion about our native languages exists. I would appreciate if readers can share their views.

Daylight Saving Time Patch from Microsoft breaks apps

2007-02-25T08:09:00.000-08:00

The Microsoft patch applied by knowedge base http://support.microsoft.com/kb/931836/ for switching the Daylight Saving Time (DST) from April to sceond Sunday of March starting 2007 caused some damage to our applications. The MS Java Virual Machine is used by us and the Java Date string format looks like "Sunday 25 Feb 12:23:39 AM EST 2007" in an unpatched server. If we pass this to Javascript to initiate a new Date object, it works.
That is, if you did Date date = new Date() in Java and applied the toString() method on it, it renders the string with timezone as "EST".
BUT, if you patched the server with the above Microsoft patch, then the Date string looks like:
"Sunday 25 Feb 12:23:39 AM GMT - 05:00 2007". So, note that now the timezone is rendered with respect to GMT. If you pass this to Javascript, it returns NaN.
That is, some change that Microsoft made to the OS in the above DST patch seems to change the way the Java Date object renders itself as a string. I do not know if this affects the Sun JRE.
How do we fix this? As of now, Microsoft has not been talking about a fix. We plan to fix it by overriding the Date object's toString method and creating our custom one.
Relevant links are in:
http://msdn2.microsoft.com/en-us/vstudio/bb264729.aspx
http://support.microsoft.com/kb/931836/
One of my colleagues contacted Microsoft and they replied Microsoft had no planes to fix this patch.
There is also information for .NET applications in the above links.

To Saral Or Not To Saral

2007-02-18T18:40:00.000-08:00

I believe there are two kinds of men – those who look forward to filing Form-16 (saral or whatever) and those who get diarrhea thinking about it.
My friend Ramesh is of the first kind – he said he feels the thrill of adrenaline as he contemplates a tax form; he knows Tax law better than most lawyers. He has a separate tax chair where he sits and pores through the ancient lore of tax evasion; loopholes; deductibles;HRA; nothing upsets him.
I, on the other hand, think deductibles are sea monsters.
I believe tax evasion has a long history – if you asked American scholars (Howard Bloom, as an example) they may trace the earliest tax evasion to Athens; and go on and on about how they fought against taxes and the Persians. In American history, everything traces back to Athens; then they move onto Rome and then jump straight to 1776.
But why do people feel the need to not pay taxes?
I interviewed an anonymous source (Ramesh) for this and he explained I am looking at this the wrong way – it is not that he is avoiding tax; he said tax loopholes are similar to the Easter Egg celebration. It is an annual game in which the government hides some statement such as “Whereas the wherewithal for hereupon the party of the first thereafter habeas corpus Ramesh no pay tax” in the 1179th page of the budget. And then they go, “Now go find it”, to all the kids. Ramesh wants to be the first to find it.
What does finding that information give him (apart from lots of money)?
To find out, I asked him, “Why do you find loopholes in tax law?”
And he said, “Because they are there”.

My company’s accounts department helpfully sent all employees an excel sheet with formulae for everything so that we could easily figure out the tax we owed.
I opened the sheet and looked at rows and rows of “HRA (18-C)” and thought, “This must be simple”.
I entered all my information and scrolled down eagerly to look at my tax liability.
It came up to 200000036749.50.
It seemed huge even to me; I experimented with adjusting the numbers and got it down to 174642397.8.
At this point I thought I needed to get out of the country fast because I seemed to be paying the most tax in the country. I had two choices – either get out of the country or go to the accounting department at 4th floor.
I steeled myself and decided to get out of the country.

I had heard horrible stories of the 4th floor accounting section – there were people who went there and never returned. Some came back permanently scarred waking up screaming from nightmares in the middle of the night.
The accounting section also sent weird emails with lots of explanations (that they probably thought were helpful). I put them in junk mail and was now paying for it (literally).
I was packing up to go to some place where there were no taxes (Moon – I know that the astronauts in the International Space Station pay taxes; it is called tax deduction from space) when a couple of other tax-challenged zombies suggested going up and talking to the accounting guys. I went up.
I have always noticed that accountants have the same smirk on their face when talking to employees as auto mechanics. Have you ever tried to explain what is wrong with your car to a mechanic? He scratches his ass when you are talking, shares jokes with other mechanics, blows his nose, spits on the ground and then blows cigarette smoke in your face. The reason is they believe you are a moron – if you do not know that the carburettor power screw nuts are slightly bent, then how good a programmer are you?
Accountants are the same way – if you do not know that section 418C is one of the three health savings account deeds under which you can claim standard exemption – what good are you? Why do you even exist? You are the reason the country is going to the dogs.
(By the way, both automobile servicing and accounting are apparently men’s jobs. I have never seen a single female accountant yet.)
But accountants do not scratch their asses while you are talking to them. You have to give it to them. They are much more sophisticated.
So the accounter guy stared at me with thinly vined contempt.
“This is your excel sheet?” he asked me, showing the printout.
“Yes”.
He surveyed it and then showed it to the guys sitting next to him. They all had a hearty laugh.
He then asked me what I paid last year.
I told him.
He smiled again and said, “I did not ask your yearly tax; I am asking about the monthly tax”.
I said, “That is the monthly tax”.
Now all of the department was paying attention to me. I smiled to myself; “so you bastards now know how important I am”.
But the accountant said, “You cannnot have these kinds of savings for that monthly tax. You need to buy a home”.
I was still smiling. “I am going to buy one soon”.
The accountant nodded, looked at the excel sheet and did some calculations. Then he said, “You are not going to be paid for the next 4 months.”

“And we need your passport.”

Thinkers and Designers

2007-02-13T15:17:00.001-08:00

I was chatting with a potential recruit to my company. He was thought to show promise by HR; he was confident, expressed himself clearly. Around 5 years of experience.
I asked him what his role was in his company. He said he was a designer. (Ofcourse, he meant software design). I asked him if I could ask him coding questions.
He gave me a condescending look and said he had not coded for the last 9 months. He purely "designed".
I asked him how he "designed" without reference to technology; he dismissed that question. He just decided how things should work. It was upto lesser "programmers" to implement his design.
He also informed me that his company had a special design team; that people were promoted to it for "outstanding" performance. They apparently stopped the lowly job of coding on reaching that team; they just designed.
I asked him specific questions on a couple of simple technologies and he had no clue. He had no knowledge about design methodologies; software development processes or any knowledge of design as a discipline.
I have always been surprised by this - how can technical people, in the beginning of their career, reject coding as menial labor and style themselves as designers? Isn't that career suicide? I would never hire a person to a technical position if they did not know hands-on coding. It is my belief that you can never really design a good software system without knowing the technologies used to some degree. Technology-agnostic design does not exist.
Does this mean a .NET architect can never design a good Java based enterprise system? It does not mean that. But it is essential to have a good grasp of atleast one suite of application technology.
As an example, let us say you are asked to create a data warehouse that aggregates data from different systems. It does help to know the concepts of Extraction, Tranformation and Loading; it helps to know multi dimensional querying; it helps to know dimensions and measures and aggregate queries. If you did not take into account specific technologies; if you have not even tried your hand at MDX; I would claim you cannot come up with a useful design.
All that you can do is draw a few blocks and say this little box talks to that big box and sound knowledgeable. CTOs can do that - not system designers. Design is more than that. It is actually creating an integrated system taking into account functional and non-functional requirements. If someone says "I sit and design a web application without being skilled in any programming language", then that person is a worthless hire.
I had the same experience with a guy in my neighborhood. He was 3 years out of college and he said his job was to "think". I am serious - he said his company pays him to think. I pictured him sitting in his cubicle contemplating the monitor. He thinks and he thinks and then he said he comes up with a solution. He then kindly shares it with the dumb programmers and they go and just do the simple job of coding. He said he did not do coding. I was surprised by this and he explained to me that he does code - sometimes - because he takes it upon himself to do a module. He was careful to clarify that nobody actually makes him code the module; he does it out of the goodness of his heart.
Actually that is what this attitude comes down to - these guys are ashamed to say they code because then that means they take orders from someone. They bow to no one. And they want to be tech leads.
That is the other aspect of interviewing - I have noticed that some people love to be questioned on "concepts". To them, a set of coding questions is insulting - one guy I interviewed in Philadelphia said it was like a quiz. I think this is just a sham attitude. If you did hands-on coding, it is the duty of the interviewer to find out. I can keep on asking vague questions about your last project, your attitude to team work, your strengths and weaknesses and crap like that - it does not help in judging your value as a technical person. A technical perosn must know coding and must answer specific coding questions, period.

Brief Introduction - OR Mapping

2007-02-07T10:37:00.000-08:00

Varun asked me some details on a database access layer. This post is an attempt to introduce developers to the complex field of Object Relational(OR) Mapping. I conducted a presentation on this subject with reference to EF internal frameworks. I have also given a set of links at the end.
For the purpose of this article let us consider that we are asked to design the middle tier and database structure of an online book store. A customer can order different books in multiple quantities. There is a Customer table; an Order table linked to the Customer; a Book_lkp table that contains the list of books displayed in the website. A Customer places multiple Orders of different books.
The Middle Tier has to provide the ability to:
1. Load a set of books for display
2. Load a Customer data
3. Save an Order for a Customer
One of the ways to design a data access layer for the above problem (in .NET) is:
Load a Dataset with the Customer and Orders and then save after making changes.
This approach is fine for simple websites - but you have to handle business logic in a set of external classes. Strongly Typed Datasets help you add type safety.
The problem with the simple approach is that for complex websites with intricate relations, your business logic will be very distributed and incoherent.
Typically your middle tier should model your business - you will need an object model. Unfortunately, databases are relational. They are not object based. For example, databases do not capture inheritance. In the above example, let us say a Customer can be a corporation also. In the middle tier, you can describe a Customer base class and derive an Individual class and a Corporate class from it. It is difficult to do this in a databse structure. Databases also do not have object identity; they do not handle relations intrinsically.
So, fundamentally, a relational database represents data and its relations completely differently from a object model. The Middle tier needs a good object model. Therefore, the problem is to map the object data to relational data. This is the problem that Object Relational mapping patterns try to solve.
One of the patterns for solving this problem is called Active Record.
Active Record
When you use this pattern, you decide that your object model in the middle tier is going to mirror the database model - that is for every table you create a class and then handle foreign key relations through child collections.
So, in the Book Store example above, you will have a Customer class, an Order class and Product class. The Customer class will have a child collection of Orders.
When you use Active Record, each class is responsible for saving and loading itself. The Customer communicates with Customer table, Order with Order table and so forth.
Commonly, in frameworks, the code for these classes are generated from the database itself. For example, a Customer class will be generated with a property for each column in the Customer table. Also, since you do not want to embed the actual mechanism of communicating with the database in each class, all these classes will derive from a base class - which handles the connections and executes the queries. (Ruby on Rails uses this pattern).
Using this pattern, you will first start with a database design and you will not spend much time on an object model design.
Since the database structure is replicated in the middle tier, the design of your object model cannot include the subtleties of a pure object model. For example, Active Record makes subclassing difficult to handle.
If you are a purist and do want to design a object model that mirrors your business instead of the database, you can use another set of patterns called Mapper.
Data Mapper
When you design an object model independent of your database model, you will still want some mechanism of saving the data to the database. Using Mapper, it is the responsibility of external classes to "map" your object structure to the database. The Mappers will handle database access and executing queries.
In the above example with the Individual and Corporate classes, you will write a base mapper for the Customer class and then derive a mapper each for the Individual and Corporate classes. The Individual and Corporate classes themselves will have no knowledge of the database. They can contain business logic; but they will be passed to their Mapper classes for saving and loading. The mappers would contain the logic to map the properties to columns. The mappers will contain the database stored procedure names, typically.
Contrast this with Active Record where the mapping is done by the middle tier objects themselves.
One way to understand this is to contrast the code given below. In the first line, a Customer object saves itself. In the second, a mapper saves the Customer object:

Customer c = new Customer();
c.name = "Ram";
c.save();

Customer
c = new Customer();
c.name = "Ram";
CustomerMapper.Save(c);

OR mapping has different shades of the above patterns. There are many open source OR mapper frameworks such as Hibernate. All of them promise configuration driven mapping; but Ted Neward and others think there cannot be a perfect mapper and that each of these need some customization. Ted Neward has even compared OR mapping to the Vietnam war here.
There are lot more details in the Wikipedia article here.
Developing your own mapper framework is not an easy task. You may be better off working with an open source framework or use Active Record with code generation.
There a whole set of patterns related to these mappers such as Lazy Loader (for loading child collections at the point of usage); Assembler (for assembling the objects); Separated Interface (for isolating the mappers). The best book reference for these is "Patterns of Enterprise Application Architecture" by Martin Fowler.

Object Oriented Javascript - an introduction

2007-02-05T05:55:00.000-08:00

Working in Photon, I get a constant stream of queries from developers about Object Oriented Javascript (OOJ). Many, of course, come across OOJ syntax in some maintenance code - and completely panic. I don't blame them - this was my reaction too when I first saw syntax as below:

Customer.prototype.getName=function(id){
return this.name;
}

What the hell does the above mean? What is prototype? Why is the function keyword placed after an equals sign?
I will present here a very essential guide to OOJ. At the end of it, hopefully you won't be scared by OOJ syntax; will be able to maintain such code properly; and even create your own classes in Javascript. To create Ajax applications, it is essential to be able to create client side object libraries.
Some of the abilities of an OO language are:
- to define classes;
- to define constructors for initialization code;
- to define methods attached to such a class;
Let us see how these work with an example. Let us say you want to create an address book. The address class has three properties - address1, city and pin code. You would like to populate the addresses from a html page. Any validation will be handled by the address class.
In OOJ, you create a class and a constructor at the same time:

function Address()
{
this.address1 = "";
this .city = "";
this.pincode = "";
}

You have defined a Address class. It looks just like a function; but Javascript knows it is a class if you use the above class like this:
var addr = new Address();
Note that you define the constructor at the same time; the above function is also the constructor. You can pass parameters to it thus:

function Address(address1, city, pincode)
{
this.address1 = address;

this.city =
city;
this.pincode = pincode;
}

var addr = new Address(addr1, city, pin);

Now, the members of this class are declared within the class. This is similar to C# or Java:

var addr = new Address();
addr.address1 = "1 second street";

You can refer to member properties using the "." operator.
So, we have defined a class; a constructor; an some properties. Now it is time to attach some functions. Let us say there is a function to validate the pincode; there are two ways to attach it.
The first is:

function Address(address1, city, pincode)
{
this.address1 =
address;

this.city =
city;
this.pincode = pincode;
this.validatePin=function()
{
var re = /\d{6}/g;
return re.test(this.pincode);
}
}

So, validatePin is a method of class Address. To use it:

var addr = new Address();
addr.pincode = "600020";
addr.validatePin();

You can handle parameters too.
The second way to attach methods is using the prototype object. In Javascript, every object has a internal hidden property called the prototype. You use the prototype property to attach methods to a class:

function Address(address1, city, pincode)
{
this.address1 = address;
this.city =
city;
this.pincode = pincode;
}

Address.prototype.validatePin = function()
{
var re = /\d{6}/g;
return re.test(this.pincode);
}

The weird syntax of functions confuses people. But, in Javascript functions are first class objects and can be declared anonymously.
In both the above examples, note that you refer to other properties in the class using the "this" keyword.
What if you wanted to call a method from another method?

var addr = new Address();
addr.pincode = "600020";
addr.validate();
Address.prototype.validate = function()
{

return this.validatePin(this.pincode);
}
Address.prototype.validatePin = function(pin)
{
var re = /\d{6}/g;
return re.test(pin);
}

The most common mistake in OOJ programming is to forget to use the this keyword when referring to other members. This will throw a syntax error.
The above illustrates creating and using a class using OOJ. Classes can use other classes; therefore you can create a proper class library. As an example, you can have an address collection as below:

function AddressList()
{
this.addresses = new Array();
this.currentAddress = null;
this.defaultAddress = new Address();
}
AddressList.prototype.addAddress=function(address)
{
this.addresses[this.addresses.length] =
address;
}

and so forth.
These class libraries are the basis for creating Rich Internet Applications (RIA). If you used global functions and properties everywhere, you will rapidly lose the ability to maintain.

OOJ does not have the ability to automatically handles setters and getters. It also does not have the ability to handle access modifiers such as private, public or protected. This does cause some problems from a pure OO perspective.
OOJ will work in all browsers that can handle Javascript - it is a fundamental feature of the language.

OOJ and Events
When calling methods on a class you do not have to worry much about object references. But, when you dynamically create user interface elements such as divs, you may have to handle events on them. It gets somewhat tricky to handle this:

function TextBox()
{
this.divTag.addEventListener("onkeydown",
this.keyHandler,false);

}
TextBox.prototype.keyHandler=function(ev)
{
alert(ev.which);
}

The above code will work.
I suggest that when you work with Javascript, and you use external files, consider using OOJ; it adds lots of clarity to the code.

The Multi-lingual Textbox concept

2007-02-02T04:43:00.000-08:00

A few days back, one of the PMs in the SME division in Photon got in touch with me - they had a client who was asking for a multi-language capable textbox. The idea is that an user can type in a language they do not know or are learning(such as Portuguese) ; as they type, they get a dictionary of words and their meanings. They can choose a word and it should be highlighted.
The client was technical - so he had already created a basic textbox along with several Javascript classes to manage the dictionary. The problem was he wanted a textbox in which styles can be applied. Now, the basic HTML textbox does not let you style. The solution is to use a that would be editable.
The client thought we can use a cursor gif file (that is an artificial cursor) and then manipulate it back and forth as the user types. It was obvious to me that this approach would be difficult to maintain; further, the client also wanted to manage Arabic with Right-to-Left text using this approach.
So I researched a little bit and found that the div tag has a property in IE - called contentEditable. Setting this property to "true" has the effect of making it editable. Unfortunately this property is only available in IE (and in Safari, I think). Firefox won't work with it.
Firefox and other browsers try to implement such editable controls using a property called designMode. Setting designMode to "on" has the effect of making a control editable.
This kind of editable divs are, of course, the essence of any Content Management System (CMS). For example, most CMS have a Rich Text Editor (RTE); in fact the blogger window I am typing this in, is an RTE. Most RTE out there (even Google's mail composer) use the above two options - contentEditable in IE and designMode in Firefox.
Both the above properties are not part of any DOM standard.
Because I used contentEditable, handling Arabic was extremely easy - it was reduced to setting the dir property:

divTag.dir = "rtl";

There are a few gotchas about using these properties:

In Firefox, setting designMode to "on" has the effect of reloading the document. Typically you use an iframe to host the editable region. If you set designmode to "on" in the window load event handler, it will reload the document and refire the load event and so on in an infinite loop. I managed this by using a hack - making sure the document does not load twice by using a global flag.
In IE, the markup that IE generates for the inner HTML in the editable div is very non-standard. You can read more about this here:.

You can find more details on building an RTE using Mozilla from the Midas spec

here

Team Foundation Server

2006-02-09T02:38:00.000-08:00

I recently installed TFS Beta3 Refresh in a dual-server mode and also moved the code base (including Version Control) from an existing single-server installation (in Beta 3) in a different server. The install showed gaps in Microsoft documentation and moving of the code base was extremely hairy.
Here is the proper order of installation in the data and application tiers:
1. User Accounts
a. Create 3 domain user accounts, one for the setup, second for the service and third for the reports. These should be service accounts whose passwords do not expire. These should be created prior to starting the installation, because the setup account is needed to login.
b. The setup account and service account should be Administrators in the application tier. The setup account, the service account and the reports account should be admin in the data tier.
c. Note down the passwords of all these accounts as they are needed in different steps of the install.

2. Data Tier Checklist
a. Login to the data tier computer using the domain user account for setup.
b. Check if Windows 2003 SP1 is the OS version in the data tier. If not, install it.
c. Install IIS 6.0. (This is not mentioned in Microsoft documentation but it is necessary for Reporting Services.)
d. Install .NET Framework 2.0.
e. Now, check if the local user account ASPNET is created in the computer; you can check this by going to Administrative Tools->Computer Management-> Local Users and Groups and click on Users. If ASPNET is a user, you can proceed to step f. If not, proceed to step e.
f. Open a command window and navigate to C:\Windows\Microsoft.NET\Framework\. Run (from the command window) aspnet_regiis –i. This installs ASP.NET 2.0.
g. Start the Microsoft SQL Server 2005 install. Follow the install as suggested in Microsoft documentation (for TFS). Make sure to check Reporting Services in the list of services to install.
h. When prompted for the Reporting Services account choose “Use the built in System account” option. For the rest choose default options.
i. After install try accessing: http://localhost/ReportServer (assuming you installed in the default website). If it does not throw an error, continue.
j. Now you are read to install TFS (databases)
k. Follow Microsoft documentation for installing TFS (databases).

3. Application Tier Checklist
a. Login to application tier computer using the domain user account for setup.
b. Check if Windows 2003 SP1 is the OS version in the data tier. If not, install it.
c. Install IIS 6.0
d. Install .NET Framework 2.0.
e. Now, check if the local user account ASPNET is created in the computer; you can check this by going to Administrative Tools->Computer Management-> Local Users and Groups and click on Users. If ASPNET is a user, you can proceed to step f. If not, proceed to step e.
f. Open a command window and navigate to C:\Windows\Microsoft.NET\Framework\. Run (from the command window) aspnet_regiis –i. This installs ASP.NET 2.0.
g. Install Windows Sharepoint Services SP2. After installation, to reboot (even if it does not prompt you).
h. Check if WSS was installed correctly by navigating to Administrative Tools->Sharepoint Central Administration. If a web page comes up with configuration information, continue.
i. You can start the install of TFS (services) and follow it as given in Microsoft documentation.
j. At the end of the install, open Administrative Tools->Internet Services Manager. Expand the websites node and check that there is a web site called Team Foundation Server.

This completes the installation of Team Foundation Server. TFS can be managed using Team Explorer, which is part of the install package. Team Explorer can be installed either in the application tier computer or in a development computer.

Note: The only administrator account by default in TFS, after a fresh installation, is the domain account for Setup that we use during installation. Team Foundation Server administrators are a separate group from computer (server) administrators.

4. Client Tier Checklist and Information
a. Install Team Explorer by choosing client tools from the installer.
b. Now, ping the application tier computer mentioned above by name. Make sure it resolves.
c. For testing Team Explorer, open Visual Studio 2005. In the Tools menu, choose “Connect to Team Foundation Server”. Add the application tier server name.
d. For testing permissions etc, right click on the server (in Team Explorer) and choose Add an Existing Project and see if you can browse existing projects)
e. An administrator needs to add other users to a project or as admins by right-clicking on the server name in Team Explorer and choosing Team Foundation Server Settings -> Security->Group Membership.
f. For testing if the interaction with Sharepoint and Reporting Services works, after the above step, right click on the server (in Team Explorer) and choose New Team Project (This will only work if you have permissions to create a new Team project. Otherwise, choose Add an Existing Team project). Type in a test project name in the wizard and try creating it.
g. The project templates are big and over slow connections, it takes some time setting up a new project.

Version Control Move
Moving from a single server to dual server was extremely difficult. In my case, the existing server was also in a different domain.
There are a couple of Microsoft documents that talk about this:
Help in migrating from Beta 3 to Beta 3 Refresh
This one can be followed mostly, except that it applies for migrating on the same server - it does not apply when you totally shift the data tier and application tier servers.
This document deals with shifting the data tier server.
Here are a few pointers:
The Source server is the existing server. The Destination server is the new server.
When you restore the databases in the Destination server, make sure that the Application Tier computer has its IIS service shut down and Stop the TFSServiceScheduler service.
Before restoring the databases in the Destination server, make sure that you take proper backups of the existing (default) databases in the Destination SQL server. I just renamed them.
Restore the Sharepoint databases first. The STS_CONFIG database is the configuration database for Sharepoint. The STS_[servername]_1 database is the content database.
When you shift the databases, the STS_CONFIG database tables still have data pointing to the old server. There are not many tables, and going through them and fixing them may help. I don't know of any scripts that help in this process. Another choice is to keep the STS_CONFIG database as it is, and move just the Sites table from the source database. The problem is, the Sites table has a foreign key from the Database table. So you may have to copy the database key, paste it and then insert
into the Sites table.
After restoring the content database, you have to open the Sites table in the content database and repoint the Urls to the Application Tier server.
The most important thing is getting Sharepoint working. You can verify if it works when you type in http://[apptier server name]/sites/[project name]. If it throws an error, you got a problem. The other indicator that Sharepoint has an issue occurs when you choose Sharepoint Central Administration from Administrative Tools in the application tier computer.
Some of the issues could be related to security. Check the Security users and roles in the STS_CONFIG and the content database. For reference, check the renamed (older default) databases.
Once Sharepoint works, restore the other databases (except for TfsWarehouse). Remember to keep the service stopped in the application tier computer.
After you restore all the databases, the TfsIntegration database needs to be repointed. There are four tables:
tbl_registry_entry
tbl_extended_properties
tbl_service_interface
tbl_subscription
After changing them and restoring security, try connecting using Team Explorer.
If Sharepoint works and Security is duplicated, the application should work.

VI Editor - common commands

2005-11-22T17:56:00.000-08:00

I found an excellent reference in Wikipedia for common vi commands. Here is the link:
VI Common tasks

Vi has two modes - Insert and Command mode.
Switching between the modes:
In command mode, typing "i" takes you to Insert mode, where you type in the text you are so interested in editing.
In Insert mode pressing the Escape key takes you to the command mode.
vi opens in command mode by default.

The vi command mode is pretty complex so I am just listing the most common things you may need to do, if you had the misfortune of having nothing other than vi.

1. Opening an existing file called ram.txt:
Type in vi ram.txt
2. Opening the same file in read only mode:
vi -R ram.txt
3. Navigating to the end of the file you just opened. Remember, you are in command mode:
Type in "G" in command mode. It takes you to the end of the file.
Type in 10G to move to line 10.
4. Quitting vi
In command mode, typeing :q! quits without saving your changes
In command mode, typing :wq quits after saving, if an existing file is opened
5. Getting Help
:help brings the help window. Typing :q on it closes the help window.
6. Navigate up and down the file
In command mode, h moves left, j moves down, k moves up and l moves to the right. Note that these keys are adjacent in your keyboard.
7. Copy and Paste
Copy till end of file, type in :yG
To copy a line, type in :y$
To copy till the next occurence of a pattern, say "END" use
:y/END
To paste, type in :p
To cut till end of file, type in :dG

To mark (block a set of lines) and then copy, reach the line where you want to mark and type (in command mode) m such as ma
Now to copy upto the bookmark, type in y' such as y'a. This copies a block of text.

8. To undo, type u
9. To search for a pattern
/END will find next occurence of END.
// will repeat the search.
10. Repeat a command
. (period) is for repeating a command
11. Delete text
dw deletes a word.
d$ deletes till end of line
x deletes current character.

The one thing I have not found is how to search a pattern and replace all occurences of it.
The Linux equivalent of vi is vim. It comes with enhanced features.

GREP for windows

2005-10-27T01:07:00.000-07:00

I found the open source Grep for Windows application from sourceforge.net URL:
http://gnuwin32.sourceforge.net/packages/grep.htm
The install does not add grep.exe to the path in windows; I had to do it myself.
The documentation has examples and all the usage options.

C:>grep --help shows all the options and their uses.
I tried some different grep statements:
1. Find Grant statements in a .NET database project
grep -E GRANT -r -h *.*
-h is to eliminate preceding filename
-E is for the pattern (extended regular expression)
-r is for recursive
I used -E because that was the first one I found, but -G can be used too.

2.Find one word with another word in the same line, irrespective of order
For example, to find the words "Emacs" and "gnu" in the same line.
grep -r -i -E Emacs *.* | grep -E gnu

This actually uses the "pipe" or "|" character to feed the output of one statement to another.

3. To find lines with (Emacs AND gnu) OR (Emac AND org)
grep -r -i -E "Emacs" *.* | grep -r -i -E "gnu|org"

The pipe "|" in this case is part of the regular expression and can only be used within the double quotes. If it is used without the double quotes, the command shell would interpret "org" as a command and not part of the regular expression.
4. Exclude a file
grep -r -i --exclude-pattern.txt -E "Emacs" *.* | grep -r -i -E "gnu|org"

excludes pattern.txt from the list of files to search.

5.Get more lines AFTER the matching line
grep -r -i -A 5 --exclude-pattern.txt -E "Emacs" *.* | grep -r -i -E "gnu|org"

The -A 5 gets 4 more lines AFTER the matching line.
The -B 5 gets 4 more lines BEFORE the matching line.

I hate running in the same place

2005-10-23T21:48:00.000-07:00

I started on my new adventures in wonderland two years back and I am still in the same place. ASP.NET 2.0 is coming out in a couple of weeks with SQL Server 2005. I know nothing about them. Macromedia Flash Professional 8.0 is coming out with regular expression support and apparently Macromedia Flex will have a new version. And I have not even started on Flex...I will wait till the next version... or the next.
If I stop running there are a couple of billion Indians, Chinese, Filipinos and Brazilians waiting to jump on me.
Here are a list of things I will try to write about in this blog.
1. ASP.NET
2. HTML, DHTML
3. Web Services
4. Code generation using CodeSmith for ASP.NET domain layers
5. Code generation using CodeSmith for Java domain layers
6. Flash programming
7. DTS
8. SQL Reporting Services
9. MDX
10. Analysis Services
11. Adobe Photoshop Image editing
12. Adobe Illustrator
13. Java Swing
14. Java Applets
15. Java Server Pages
16. EJB
17. Linux Basics
18. Microsoft Application Blocks
19. ADAM, Active Directory
20. Windows Installer
21. XML, XSL, Schemas and XPath
22. XHTML
23. XQuery
24. Model based development
25. Visio
26. Design Patterns
27. PHP, MySQL development

I know enough about some of these to write; I don't about some others and I will learn as I go.
This blog is meant more as a personal refresher and goal setter than an authoritative source - because I am not really an authority. If you want an authority, get those huge WinForms books by Petzold.
Let me see if I post atleast one ,more entry in this blog...