Amazon SimpleDB in Nutshell for those who know RDBMS Systems

Note: Latest Available Amazon SimpleDB version as of today, i.e. when I am writing this, is in Beta status with API Version: “2009-04-15”

Overview of SimpleDB
Amazon SimpleDB is a highly available, flexible, and scalable non-relational data store that offloads the work of database administration. Developers simply store and query data items via web services requests.[1]

Representation of Data with SimpleDB:

Domains— Domains are similar to tables that contain similar data.

You can execute queries against a domain, but cannot execute queries across different domains.

Attributes— They are similar to columns in RDBMS, attributes represent categories of data that can be assigned to items.

Items— Represented by rows, items represent individual objects that contain one or more attribute name-value pairs.

Values—Similar to column value, values represent instances of attributes for items. An attribute can have multiple values. There is no data typing supported for attribute and all data is treated as text data during query execution.

However, Amazon SimpleDB is not a relational database, and does not offer some features needed in certain applications, e.g. complex transactions or joins (i.e. execute queries across different domains).[1] You need to rely on duplicating the data to avoid such scenarios.[2]

Benefits of using SimpleDB

Highly Available
Amazon SimpleDB creates and manages multiple geographically distributed replicas of your data automatically to enable high availability and data durability.
Flexible
You can change your data model on the fly, and data is automatically indexed for you.
Scalability
You can access additional machine resources by spreading your data set and requests across multiple domains.

Support for Reporting
Sql Server Reporting Service (SSRS) is a popular platform to build and access reports. SSRS reports are build based on dataset that has set for a report. It is possible to build dataset from SimpleDB.[3] 

Limitation with SimpleDB

Domain (similar to table): 250 active domains per account. More can be requested by filling a form. Note that each attribute (similar to column) can hold multiple value.[4]

Attribute (similar to column) name-value pairs per item: 256[4]

Maximum response size for Select: 1 MB (large data like images can be stored separately into cloud as files and an Attribute can store URL for the resource).[4]

Maximum items per select: 2500[4]

Attribute value length: 1024 bytes[4]

No datatyping: text only. Integers and reals must be represented using leading zeros to ensure proper query comparisons.[5]

References:

[1] Amazon SimpleDB (beta)
http://aws.amazon.com/simpledb/

[2] How And Why Glue Is Using Amazon SimpleDB Instead Of A Relational Database
http://blog.getglue.com/?p=1145

[3] Accessing SimpleDB from SSRS
http://www.chrisumbel.com/article/simpledb_ssrs.aspx

[4] Amazon SimpleDB Limits, Amazon SimpleDB Developer Guide (API Latest version)
http://docs.amazonwebservices.com/AmazonSimpleDB/latest/DeveloperGuide/

[5] M/DB - A Free Open Source "plug-compatible" alternative to Amazon's SimpleDB database
http://gradvs1.mgateway.com/main/index.html?path=mdb

How Apple's Location Services Work

Even though I like apples (specially the one which fell on Newton's head), I am not found of Apple as a corporation. Mainly because it is too secretive in its technologies and methods of doing something amazing. Location Services are one of those secretive things which need to be so secret but you can figure-out how it works.

Apple's Locations services uses (in order) A-GPS, Wi-Fi triangulation and Cell Triangulation (base on capability of your device).

Uses A-GPS
Pros
When you are outdoors, works amazingly.
Cons

Works poorly in urban environment with tall buildings. Even your vehicle roof is a problem for A-GPS. Doesn't work without GPS chip in device (case for iPod).

Uses cellular tower triangulation
Pros

Even with it's worst accuracy, sometimes it is more accurate than GPS in urban environment. This is because performance doesn't decrease in urban environments due to many towers triangulating in cities nowadays. 

Cons
Cellular tower triangulation as it is way too in accurate for the accuracy need of the day. This is option only when when no other medium of deciding location available.

Uses Wi-Fi triangulations
Pros
Works very accurate when one or more wi-fi is available in area. You need not to connect to a Wi-Fi, nether the Wi-Fi needs to know your requirement of locating your device. If the location of Wi-Fi is stored in Apple's giant database of all Wi-Fi and their co-ordinates, you are in luck. Your device contact Apple's service and based on Wi-Fi details, Apple will send you Wi-Fi coordinates. I am not sure if device triangulates the Wi-Fi once the coordinates are found or Apple's service will do it - but end result is accurate coordinates.
Cons
You need to be in range of one or more Wi-Fi. If you are using iPod, you should be connected with internet.

How Does Apple Get all Wi-Fi Locations?
Hmm... so the world need to have a database of all wi-fi and their locations. It is still guess for the people how apple has got this database? This is what I think based on recent controversies regarding Apple's privacy policy regarding sharing your iPhone location with Apple's Service:
When your device has correct location decided, it also looks for wi-fi around your device. It stores those information on iPhone and iTunes on your computer sends it to Apple. And so, this list has grown 2 fold in day and 4 fold in night (it is just saying).
Note that before to iOS 3.2, this was not the case so Apple used to use (and still using for older devices) Skyhook Wireless and Google's services for WiFi triangulations.

Now you are ready for next level: Go to Apple's Office Discloser of its Location Service and know more about it.

Good User Experience

Good user experience cannot be achieved by jazzy animations. It is these simple things that everyone wants to have.

Some common tips for better user experience (most of us knows but we don’t follow):

1. Use standard controls. In laymen language, text should look like text and button should look like a button.

2. Optimize for speed. Everyone want to get the things done faster. Don’t put animations which hinders speed of doing something. Your UI (specially forms) should be accessible by ‘keyboard only’ as well.

3. Think about the common mistake that user may be making while data entry. When you find something wrong, locate exact position and show understandable message to show the error.

4. Try to optimize available space. You controls should be able to adjust to larger width to avoid scrollbars if possible. Don’t put big logos of the application on each screen top.

5. Find the most used flows for your UI and optimize UI so user can complete the common tasks faster. If necessary, create different views of the same UI for different kind of user in organizational hierarchy.

Nested Transaction Handling with NHibernate

NHibernate doesn’t have inbuilt support for nested transaction (or I am not using latest version). But, with little compromises, you can create a class that enables almost everything what you really need. I am using NHibernateSessionManager which you can find here: http://www.codekeep.net/snippets/8b94e3e0-3ffd-4b59-b6ce-ed4d46158a7c.aspx

    public class SmartTransaction
    {
        bool _transactionOwner =false;
        bool _transactionActive = false;

        public bool IsActive
        {
            get
            {
                return _transactionActive;
            }
        }

        public void Begin()
        {
            if (_transactionActive) return; //return to bad programer's code

            try
            {
                _transactionOwner = !NHibernateSessionManager.Instance.HasOpenTransaction();
                if (_transactionOwner)
                    NHibernateSessionManager.Instance.BeginTransaction();
                _transactionActive = true;
            }
            catch
            {
                _transactionOwner = false;
                _transactionActive = false;
                throw;
            }
        }

        public void Commit()
        {
            if (!_transactionActive || !NHibernateSessionManager.Instance.HasOpenTransaction())
            {
                _transactionActive = false;
                return;
            }

            try
            {
                if (_transactionOwner)
                    NHibernateSessionManager.Instance.CommitTransaction();
            }
            catch //when you don't know what you are committing to.
            {
                NHibernateSessionManager.Instance.RollbackTransaction(); 
                throw;
            }
            finally
            {
                _transactionActive = false;
            }
        }

        public void Rollback()
        {
            _transactionActive = false;

            if (!NHibernateSessionManager.Instance.HasOpenTransaction())
                return;
            NHibernateSessionManager.Instance.RollbackTransaction();
        }
    }

Benefits:

1. When writing a method, you need not to worry about context of calling method. You can just begin transaction of SmartTransaction class. If calling method has started transaction already, SmartTransaction doesn’t begin the transaction, but if something goes wrong, it will rollback entire transaction (of calling method).


2. You need not to worry if method you are calling needs transaction wrapping. As with SmartTransaction class, if method you are calling needs transaction wrapping, it will have transaction wrapping using SmartTransaction class.

iPhone Button for Navigation Bar

If you are developing iPhone application and trying to use UIButton on ‘Navigation Bar’ instead of ' ‘difficult to use’ ‘Bar Button Item’, you need a button image that exactly looks like Bar Button.

As I couldn’t find the button background on net, I created it myself using Paint.NET. Here it is for  you.

What are Characteristics of Successful Innovative Idea

I’ve been associate with an Innovation Lab from quite a some time and observing for life how a genuine ideas turn into a success. Though I’ve never been able to own a killer existing idea, I can still depict some characteristics of successful innovative idea (based on guts but without example or stories - bad bad to convince people).

• Idea that solve problem for many.
• Idea that have feasibility to be prototyped.  Idea that can be prototyped in short duration.
• Idea that reasons (in term of Math, Science AND Art) why the idea can be successful. Idea should reference similar success.
• Idea that is inline with what your associated firm is trying to do.

Concatenation of Styles in WPF

In order to avoid confusion and/or be inline with OOP concepts, WPF style will not allow you to concatenate multiple style in one control. Instead, you can use ‘BasedOn’ property (of Style element) to inherit from the other style. It is possible to create chain of inheritance but multiple inheritance is not supported by default. Initially I though it is not possible  until I came across this amazing link: http://swdeveloper.wordpress.com/2009/01/03/wpf-xaml-multiple-style-inheritance-and-markup-extensions/

It shows use of MarkupExtension to enable inheritance from multiple styles (in actual, your triggers and setters will be merged. Duplicates will be overwritten by the second style that you define)

Happy interfacing!

Solve “You are not allowed to upload ‘FileName’. the requested file type: 'FileType’” problem on Vanilla Forum 1.x

Issue:

You get this exception when you try to upload any file in your Vanilla Forum (Version 1) using any Attachment extension :
You are not allowed to upload ‘FileName’. the requested file type: 'FileType’

Resolution 1:

Add specified type as allowed type. Steps to do so follow here:

1. Note the 'FileType’ in the error message.
2. Go to extensions\Attachments\default.php file in your Vanilla forum’s directory.
3. Add appropriate entry in Context->Configuration['ATTACHMENTS_ALLOWED_FILETYPES'] list.
4. Note that it’s also important that you give correct extensions for the mime type you are allowing as both, 'FilteType’ (i.e. mime type) and file extensions are corss checked in Framework code of vanilla forum.
5. Save the file and test.

Resolution 2:

Remove check for allowed types. Note that this is not very safe for public forum but it’s ‘ok’ for internal forum with known and responsible user. Steps to do so follow here:

1. Go to “library\Framework\Framework.Class.Uploader.php”  file.
2. Find this line of code “if (!array_key_exists($FileType, $this->AllowedFileTypes))”  and comment this ‘if’ and ‘else’ of it. Yes you need to comment if AND else.
3. Save the file and test.

Solve Exception Message: The IAsyncResult object was not returned from the corresponding asynchronous method on this class

Recently I was got a weird error on my Windows CE code (.NET compact framework) which was asynchronously listening for UDP (or may be same for TCP) messages.

Exception Message

Error Message: The IAsyncResult object was not returned from the corresponding asynchronous method on this class.
CallStack for debug purpose:
    at System.Net.Sockets.Socket.EndReceiveFrom()
    at BallyTech.SocketListener.UdpSocketListener.OnReceive()
    at System.Net.LazyAsyncResult.InvokeCallback()
    at WorkerThread.doWork()
    at WorkerThread.doWorkI()
    at WorkItem.doWork()
    at System.Threading.Timer.ring()

Issue

When frequently Open/Close UDP sockets, above exception comes while closing one socket and start listening on another socket on the same port. OnReceive method is called (only once) for the previous (closed) socket while doing “Start Listening” on new socket on the same port. As old socket is already dereferenced and closed, you will get above socket exception on EndReceiveFrom or EndReceive method.

Solution

In short, you have to ignore the calls of your OnReceive (or similar) for Old socket which you no longer use i.e. closed sockets.

Note that when you call BeginReceiveFrom to start listening for messages, it returns an instance of IAsyncResult. Don’t ignore this and store it into class level private variable (let’s say variable name currentAsyncResult).

Now make modification into  your OnReceive method to ignore any message which is not for currentAsyncResult.

private void OnReceive(IAsyncResult ar)
{
   try
   {
      if (ar == currentAynchResult)
      {
         IPEndPoint ipeSender = new IPEndPoint(IPAddress.Any, 0);
         EndPoint epSender = (EndPoint)ipeSender;

         //Error comes here if we didn't have (ar == currentAynchResult) check
         int bytesRead = udpSocket.EndReceiveFrom(ar, ref epSender);
         //process further
      }
      else
      {
         //Ignore
      }
      BeginReceive();
   }
   catch (Exception ex)
   {
      //Log exception. Don't throw exception. Most probably BeginReceive failed.
   }
}

Notice the if (ar == currentAynchResult) check where we got currentAynchResult from BeginReceiveFrom method while starting listening from new socket.

Happy Networking!

Configure OpenCV 2.0 on Visual Studio 2008 for VC++

You must be hating OpenCV 2.0 by now :) But don’t worry your pain will go soon (if yes, please write a comment). The biggest pain is that you will have to compile your OpenCV using CMake and all those stuff (as explained here). But if you are content with C or C++ and not interested in debugging inside OpenCV or Python then read further.

Download and Install, Emgu which is actually .NET wrapper for OpenCV but we will just use their precious .lib files which otherwise is pain in neck to generate (Download Emgu from here: http://sourceforge.net/projects/emgucv/)

After Emgu installation, Open Visual Studio and follow these steps:

1. Go to Tool –> Options –> Projects and Solutions –> VC++ Directories

2. Now see a combobox “Show Directories for” and select “Include files” and add, “C:\Program Files\Emgu\Emgu CV\opencv\include\opencv” (verify, as program file location may differ if you are using 64 bit operating system)

3. Now in the same combobox select “Library files” and add, “C:\Program Files\Emgu\Emgu CV\lib\release”

4.  Now go to your VC++ project, right click and go to Properties –> Configuration Properties –> Linker –> Input –> Additional Dependencies and add these lib files (which is at  C:\Program Files\Emgu\Emgu CV\lib\release) cv200.lib cvaux200.lib cxcore200.lib highgui200.lib (check library name, it may change according to OpenCV version used for the Emgu)

5. That’s it. Done for now. Take shot at compiling your VC++ code downloaded from internet ;)

NOTE: If you are interested in going into OpenCV library code while debugging then this may not be the best solution for you as Emgu only gives release .lib files. You will have to generate ‘debug’ lib files as described in http://mirror2image.wordpress.com/2009/10/20/switching-to-opencv-2-0-with-vs2005/

I am not sure but this same should help to configure VS 2010 also.

Wondering about System.Int32 implementation

If you are a .NET programmer (and do not use reflector), you would look at int and System.Int32 as same. But if you are also interested in reverse engineering, you would surprise how would they implement something like this.
This is how Int32 (part of mscorelib.dll) may look like in reflector,

namespace System
{
//attributes blah blah blah
public struct Int32  //implements blah blah blah....
{
internal int m_value;
//… other code with methods with ‘int’ as return type or parameter
}
}

But if int and Int32 are same, wouldn’t it become circular reference? (not ‘reference’ exactly has int is value type ;). The answer is NO. as we use them in real life. It seems reflector’s code is hardcoded to show it or mscorelib is compiled in some alien world.

I read[1] that there a special treatment given for 'int' (and so all other keywords). Interestingly, I learned creating my Int32 (and other predefined types). When I created something like this, in my project,

public struct Int32
{
public Int32 m_value;

public Int32 GetMe()
{
return this;
}
}

But after compilation, I saw (in reflactor) compiler did this to my class,

public struct Int32
{
// Fields
public int m_value;

// Methods
public unsafe int GetMe()
{
return *(((int*) this));
}
}

hmm.. Interesting. It looks like (also suggested in link [1]) when the compiler encounter an 'int' or 'Int32' in the structure which cannot be replace by pre-compiler as Int32, it gives 32 bit to variable m_value there. So, basically variable m_value doesn't exist as int or Int32 but it is just 32 bits which can be retrieved using unsafe code as shown above. Combination of visual studio intellisence, pre-compiler, compiler are specially designed to handle this scenario.
Note: For losers using Java, int is value type and Integer is reference type. Both are not same there. What a shame!

[1] C# compiler magic regarding int vs System.Int32 Thread on MSDN

Fix "The project type not supported by this installation" blah blah blah after resetsettings command

Is your visual studio settings are messed up? You can reset your visual studio settings by starting it from visual studio command prompt:

devenv.exe /resetsettings

But often, WCF related stuff doesn't work after above command. When you open the WCF porject in your Visual Studio, you may get this error:
"The project type not supported by this installation" blah blah blah..

If you are the one who has done so, close the Visual Studio and run this command from visual studio command prompt:

devenv /ResetSkipPkgs

It should reset (i.e. remove in our case) the checks that should be skiped in order to open newer WCF project into your relatively older visual studio.

.NET Namespace Naming Convention with Team Name in it

I hated naming conventions from collage time and nothing is much different today. Saying that, I also must accept that I secretly liked Microsoft's guidelines about naming conventions as it gives more freedom and meaningful restrictions than others (specially Sun Java). Let's leave other naming conventions and concentrate on namespace's maming convention of your assembly and classes.

Why namespace naming is important?
A namespace gives first hand impression on what the class and its method must be doing. With that, a class should get a unique identification based on its namespace. Carelessly crafted names are misleading advertisement which repulses intended audience or create confusion.

How Microsoft Tells us to name a namespace?
CompanyName.TechnologyName[.Feature][.Design]
is what Namespace Naming Guidelines tells. Example being Microsoft.Build.Tasks

Is it long enough?
Organizations, with help of smart programmers, are now taking little more effort to refactor the code to extract common code which can be reused across teams. It is not impossible (infact it's more frequent) to have multiple implementation approach of a 'similar' feature by different team. For example team 'TreamA' and 'TeamB' in organization 'MyOrg' might have taken differnt approach to develop a media content viewer optimized for their own scenarios. If the organization want to share these media viewers across many teams to use, 'unique identification' functionality of namespace stands violated as media viewer class from both the team will look like,
MyOrg.Media.Viewer
This would not have occured if the namespace of viewers were named "MyOrg.Media.TreamA.Viewer" and "MyOrg.Media.TreamB.Viewer" beforehand.

So?
I believe we should giving implementing team name in name as ending part of namespace. So my format would be
CompanyName.TechnologyName[.Feature][.TeamOrGroupName][.Design]

How to Fix Error 1606: Could not access network location

This error often comes while installing some software.
Scenario A: You are getting error 1606 saying "Could not access network location [Some environment variable with % in it]..." then these links may help you solve the issue.
http://support.microsoft.com/default.aspx/kb/330766
and
http://support.microsoft.com/kb/256986/

Scenario B: You are getting error 1606 saying it cannot access internet location. But you can download same file using internet explorer,
1. Try hitting 'Retry' button. You may be in luck.
2. Not solved yet? Most probably you are behind proxy server. To solve this issue, go to Control Panel -> Internet Options -> Connections -> LAN Settings. What do you see? Is "Automatic Detect Setting" checked? If yes then uncheck it and see if your installation works. If it still doesn't work, contact your network administrator and manually provide proxy IP and Port in the LAN Settings of Internet Options.

Still not working? Please find solution and put your solution here :)

Get Exception without being in catch block

In .NET, when exception is thrown, the exception passed though the call stacks. While debugging you may not be inside the catch block (may be in finally) or you may not have catch block at all. But if you want to still see what exception occurred then just go to 'watch' window in visual studio and type '$exception' and you will get the exception as it is.

Happy Debugging!

ps: I got this trick free from my ex-colleague Manish

Syntax Highlighter on Blogger

At last, I got source Syntax Highlighter on my blog. Adding script in Gadget didn't work for me as I couldn't add <link> tag in HTML/Javascript gadget. Then I tried adding script in template itself and it works like a charm :)

function test() : String
{
     return 10;
}

Interoperability Problem with WCF Web Service Solved

I've spoken to many of my friends about how WCF services do not work with older system (and some time modern system like Flex). Well it is my bad that I didn't search for a solution. Sorry! I didn't search for the 'problem' itself.
Now, when I was generating WSDL file from my .svc, I good a strange thought. I didn't want to give 1 WSDS and 3 xsd imports (yeah 3 xsd:import.. see your WSDL), I wanted to flatten it a bit and give single WSDL. While searching for the solution, I found this article by Christain Weyer. It revealed to me that lack of 'single and complete' WSDL file is the problem for old consumers. I highly encourage you to visit his blog and see the solution by yourself.

Wix with util:User

Wix installer is my latest adventure. It's good but few thing may leave you scratching you head for hours. One of them is user right assignment.
I wanted to give ASPNET user some access right to my MSMQ. So defining user went like this (after that I gave permission):

<util:User Id="aspnet" Name="ASPNET" / >

Everything worked fine, until my product was uninstalled (for testing... in real life people will love it). Uninstaller 'owned' the ASPNET user and deleted while uninstalling the product. Leaving system corrupt. I had to rescue my system's dangled processes by re-registering ASP.NET. And found (hit and try) this is the right way:

<util:User Id="aspnet" Name="ASPNET" CreateUser="no" UpdateIfExists="no" />

I don't know much about it but this worked and 'happy ending' is all I want at the starting of my weekends :)
Happy Weekends 2 u 2!

Don't flush the Session after an exception occurs

I recently moved my all 'flush' at central location i.e. on session closed. And I close session in "finally" block of the code, to ensure that it really gets closed (not really).
So code would look like

try
{
//Use Session like never used before
}
finally
{
NhibernateSessionManager.CloseCurrentSession();
//closes session that was assigned to the thread
}

CloseCurrentSession in NhibernateSessionManager would look like this,

CloseCurrentSession()
{
ISession session = GetCurrentContextSession();
if(session != null && session.IsOpen)
{
session.Flush(); //evil is here
session.Close();
RemoveCurrentContextSession();
}
}


I was proud of this solution until I got into a database exception in my main code (which happens rarely after I write millions of lines of code).
If any such exception occurred and you try to flush, you will be delighted with another exception when you use session.Flush() It says "don't flush the Session after an exception occurs". For undisclosed reason, I couldn't use autoflush (reserved for next blog may be ;)

I believe there are alternate solutions but I as I told, I am lazy (like any other ORM), so I reverted back to my 'Flush manually' everytime when you make some entity dirty. Pretty shitty stuff huh???

Anyways, thanks for reading the blog even though the title "Don't flush the Session after an exception occurs" tells it all and inside matter is just ramblings.

Caching Problem with XMLHttpRequest

Using XMLHttpReuest is fun. But if you have just started the fun, you should know this.
IE has a known issue which caches the data if request is same. It's real mystery for me why Microsoft ended up with such default behavior for functionality which is responsible to load dynamic data. But as we have to live with it, here is a small hack which will turn off the caching for that page. You need to add this two lines in your HTML Head tag.

< meta equiv="Pragma" content="no-cache"/>
< meta http-equiv="Expires" content="-1" />

Solution Source: http://en.wikipedia.org/wiki/XMLHttpRequest#Workaround