Insourcing for Development – A Networks of Action Approach to GSD

When presenting “The Research Agenda for IT impact sourcing”, Heeks places Ethical outsourcing and Social outsourcing within the scope of what can be broadly referred as the use of outsourcing for development. In the BoP (Business Operation Process) outsourcing continuum he differentiates these from Exploitative Outsourcing and Commercial Outsourcing. You can read about these terms in depth on his blogs. The focus of Social Outsourcing is on contracting out goods and services into social enterprises. In their paper, Heeks & Arun (2010) highlight that social outsourcing has the potential to deliver development benefits to marginalized groups.

In the IT impact sourcing model the idea is to create sustainable jobs in communities where opportunities are low and can act as income improvements. While this is a useful and more common way to look at developmental impacts of outsourcing, there is another way to use IT for Development. And that is through the view of offshore insourcing.

Global Software Development (GSD) is a fairly common practice in large software projects. As an arbitrage in globalized markets, it is more common practice to contracting with a wholly owned subsidiary located in another country. This is offshore insourcing. While insourcing in itself might be offshore or in-country and there has been a recent push at GM towards insourcing, offshore insourcing has many advantages that are seldom described in research. This is what I’d like to add to the research agenda for IT impact sourcing.

Titlestad, Staring and Braa (2009) highlight how the design of health information in the global south has been co-ordinated in a GSD fashion. The project’s core development on what is referred to as global release happens at the Univ of Oslo, Norway, but the requirements for this comes from different local teams based in the “global south”, as can be summarized from their paper below:

Since this is an open-source project (DHIS2), the idea of wholly-owned subsidiary might be unconventional to think about, but being part of the same global research network (HISP), each of the local nodes actually act similar to what would happen in a large global software corporation. Thus, the local software requirements, design and use happens in different countries in the “global south”, but most of the global/generic software development happens in Norway. This type of offshore insourcing is done to keep “generativity” (Gizaw, 2013) of the software intact, such that it can be “ready-for-customization” and be “flexible”, which does not have the features that are specific to a country implementation. This allows new implementers or new country to use DHIS2 without any costs to software development, although only customizations to the context need to be done. Even then, the generic features are available at a much lower cost (following the principles of libre software) for the new implementations than it would, if they were to develop the features from scratch.

How the GSD model has contributed to developmental impacts in many countries in “global south” has been discussed through many research articles. Staring & Titlestad (2008) describe the global software development and commons-based peer production of DHIS2. Through practical examples of the project they discuss the software development practices that are aimed at improving public health sector in the south. Many other researchers in the HISP network over the years have shown the developmental impacts resulting from the project and its use in developing countries. This action-research approach of the HISP network has been referred to as “Networks of Action”, where the method of action research has been shown to have sustainable developmental impact in the “global south”. Combining these concepts of Insourcing for the purpose of development using the Networks of Action approach is what should become part of the IT impact sourcing research agenda.

This blog post is just to introduce the idea that instead of focusing on outsourcing alone, “IT impact sourcing” can also cover insourcing and development through insourcing. Much more detailed analysis and discussion on this will be part of an upcoming research paper.

Limits to growth?–waiting for the 40yr update

While dabbling around arguments on how development hasn’t reached everyone, especially looking at the cases in India and Gujarat, I’ve also been reading the book Limits to growth – the 30yr update (thanks dada!!). This post is mainly to say that I desire to read the next update of this masterpiece of work that has been updated over the years since it was first released in 1972. Hunter Lovins says “If you only read one book…make this it!”. I’d say the next one “40-yr update” will be more interesting, given the global financial crisis!!

Just last night I watched, Michael Moore’s exaggerated critique on Wall Street – “Capitalism – A Love Story” and that made me think how different would the world be today, if as the movie puts it, we had a love story with Democracy instead of Capitalism. Instead of Socialism, he suggests it should be called Democracy… I wonder though if economics can be simply governed through a set of political principles. I know Adam Smith talked about political economy some centuries ago, but was his perspective too simplistic to look at the globalized world that we live in today? Would we be able to label China as a non-democratic Capitalism or FDI in India as global capitalism and not Socialism? Its hard to argue for any of these labels; to look through Marx or Smith’s worldview. Because in my opinion, the complexities around globalized economies makes it much harder to box political economies any more. The US Wall street has much more effect on the farmer suicides in Maharashtra or Andhra Pradesh than what Smith or pure Capitalists or free-enterprisers would have us believe. As Meadows said in 1972, we have “overshot”. We haven’t prepared ourselves better before globalizing. The three causes as they say for overshoot are the same, whether personal or planetary -

• There is growth, acceleration and rapid change
• Some limit, some barriers beyond which the system may not safely go
• Delay or mistake in perceptions and responses that strive to keep the system in limits

Growth is believed to be necessary for employment, upward mobility and technical advancement. For places with poverty, growth is believed to provide resources out of poverty. Growth can solve some problems, but it creates others. That is primarily because of limits and the book highlights the limits very nicely. The book was written 10yrs back before sub-prime crisis, debt problems etc. that have affected nearly all of the world’s interconnected populations. Greed, overshoot, delay in perception or response, whatever you may call it… it has highlighted that there are limits to growth and how we can’t sustain this single-minded approach to achieve growth

I’m a bit sceptical on how World3 (their model) bases itself on the central assumption that population and capital are structurally capable of exponential growth. They cite examples of different countervailing pressures that they’ve added as feedback to the model, but I’m less convinced that they’ve had enough considerations on emigration, where people are moving back to growing economies as soon as things have started to slowdown in rich countries.

What is brilliantly highlighted in the book - that growth in the way in which it has happened since 1930s has created enormous disparity. A fourteenfold increase in world industrial output since 1930s has created more disparity and hasn’t ended poverty. Another fourteenfold increase (if possible with earthly limits) would not change it either if we continued in the same way… Running the system harder or faster will not change the pattern as long as the structure is not revised. And over the last 10yrs, since they wrote the book, if anything, we see increasing disparity and faster running on this non-sustainable system.

While the book and World3 model supposes many ways to collapse, it also in Chapter 7, shows a model that can lead the world to not collapse. They call this “Sustainable System”. In Chapter 8, they suggest “Tools for the Transition to Sustainability”. They mention that “Everywhere we find folks who care about the earth, about other people, about the welfare of children and grandchildren”. Not just their own, but of others. The problem is are these folks doing enough to change the model? Are we so many and doing so much as to move towards sustainability? They say the next revolution has to be Sustainability, just as the previous 2 revolutions were Agricultural and Industrial. I wonder though will all those who have not been touched by the previous 2 revolutions be able to “leap-frog” to the third revolution? What tools do we have to reach a sustainable system – Visioning, Networking, Truth-telling, Learning, Loving. Sounds like the Buddha??

Though there is no way of knowing for sure, other than to try it.

PhoneGap under Incubation at Apache Foundation

When Adobe announced the acquisition of Nitobi, the creators of the nice cross-mobile  development tool, like many people I was worried that this would become a closed-source Adobe Air mobile. Adobe has recently been voicing that it would deprecate Flash mobile and instead move towards HTML5 and JavaScript. Thus, PhoneGap seemed like the perfect acquisition to make money out of the new generation of cross-platform mobile development.

The heart-warming announcement came a couple of days back when it was decided that PhoneGap was moved to Apache Foundation as an incubation project. The project is called Callback and the code has been moved to github for now. There are also some nice tools coming in Adobe Dreamweaver which can be used to work with PhoneGap and with the acquisition it might get better and better integrated. This means that Adobe will hope to make money out of the development and probably the build service, yet keep the platform open-source and free…

All in all, I think this is great movement for PhoneGap and cross-platform mobile development. What has been hard for ages since the promise of JavaME, seems more likely now. Good job Adobe!!

Women more respected in Pakistan & Bangladesh than in India

I’ve been pretty involved in health information and monitoring the state of affairs in that domain, but this is something I believe is as critical for humanness. The UNDP just came out with their Human Development Report (HDR) and celebrating its 20th Anniversary… I would like to congratulate guys at UNDP for the great work with this report and its indeed an excellent piece of work that has been going on for the last 2 decades. Also who better than Amartya Sen introducing us to the report.

The report is one of the most complete ways to give the overall picture to development. There are obviously critiques to how development is measured and qualitative analysis is sometimes required to study the cases. Nevertheless, it is indeed rich in data for looking at how development is measured. The report covers Health (… and life expectancy), Knowledge/Education, Standards of living, Innovation, Economic growth, Empowerment, Gender, Income inequalities, Security… and creates an index comparing these at a benchmark number of 10.

India ranks 119 on the Human Development Index and highlights somewhat of a sad story, when we are so happy celebrating our growth rates and world reputation success, a lot many things are lacking. May be even aggravating - in this view of achieving only economic growth and race for money and world power! Sure, we are doing great in democracy and political freedoms, but what about the many unfreedoms of gender and income inequalities. I would say it’s a shame that women feel more respected in Pakistan than in India, although we are the world’s largest democracy in the world. More and more as you read the whole report…

Sitting in Norway and reading the HDR feels like we are miles away in India…, but even in closer geographies (e.g. SriLanka), I feel we are very far off from our potential as a society!!

“Garage model” Leprosy MIS

While the world’s managers and venture capitalists have lost faith in the “garage model” of business, I am still a firm believer that the best innovations come from individual efforts and not concentrated directional movement in large organizations. Today, was yet another day in the field showing how homebrew software has innovations that many large software suites lack.

Just to give a background on what I’m talking about - a product that I’ve been working on lately is a patient-tracking system for the National Leprosy Eradication Programme. Since DHIS2 is the widely deployed HMIS application in India and aggregated reports for the Government of India are generated through DHIS2, we needed some kind of patient-tracking, yet required aggregate numbers of these patients through DHIS2. For this purpose, the most ideal way we found out was using OpenMRS for the patient-tracking and then generating reports through DHIS2. Thus, we are able to create rich and generic medical records of a leprosy patient, which can later be used not just for leprosy patient care, but future medical treatment of the person for any other disease or health service provisioning.

But today when I arrived to demonstrate this system in Maharashtra State Health Society, I was pleasantly surprised to see a homebrew application. We finished our presentation on the system, demonstrated the application, decided upon pilot process, future meetings and the whole process before we can go live with the system state-wide. Just after we finished our stuff, one of the leprosy health officers told us that he had privately worked on something that was suited for the NLEP program. Most people just ignored his comment, but I was eager to see what he had developed.

And when I saw his application, I was very much stunned. A really nice and intuitive user-interface using Adobe Flex, neat use of data using good-looking reporting and visualization tools, basic reports using excel sheets and a neat-little data model, similar to the ideas in OpenMRS. Maybe the data-model is not suited to a medical records system like OpenMRS, but it very well met the requirements. From what I learnt, it was developed in about a month’s time by one developer, who happens to be the son of this medical officer.

“Commendable effort and nice design decisions”, is what I complemented the medical officer. But what got etched in my mind is that, some best software are written in “that car garage” on a single computer.

The Mobile Phone Tool for Indian HMIS

As you have already found out, I haven’t been blogging lately because of the travel and development work I have been doing for HISP India. One of the new developments that I have been working over the last 3 months is a mobile phone application for routine health data collection, obscurely called “Mobile-SCDRT”. Its a Java ME MIDP that runs on all those commodity mobile phones and is used by a community health Sub-Center for Data Reporting and Transmission (SCDRT). Now you know where that obscure acronym came from and I am unsure who gave it that “wanna-be” name.

That apart, the mobile application is being deployed part of a pilot in the 5 states of Himachal Pradesh, Kerala, Rajasthan, Nagaland and Gujarat. As part of the pilot 200 health workers from the above states receive a Nokia 3110c mobile phone with the application installed and use the application to report the routine health-related data that is collected by the Government of India as part of Health Management Information System (HMIS).

The following is an architectural overview of how the application works for those of you interested in the technical details:

So, DHIS2 is where the data gets added after being sent from the mobile phone and in DHIS2 you can do all the analysis and “use of data” for health program decisions. There are other nitty-gritty details of the application and lots of new learnings for me, but I’ll leave that to another blog entry for another day.

We have started the pilot in 2 states: Himachal Pradesh and Kerala. Its been running for close to a month now in these 2 states and from the looks of it, people are happy and excited to use it. Recalling an incident from the launch in Kerala:- When just about 15 minutes of looking into the application, a JPHN (Junior Public Health Nurse - as health workers are called in Kerala) could explain the whole application, its use and reporting procedure to her colleagues better than most of us could. It was a wonderful and humbling experience for us to see that people were so much wanting to use the application and learnt it so quickly. It was satisfying to see that it was helping the health worker to make her reporting process more efficient. What was more rewarding for her, as I understand now, was the job perk that she thought she got in the mobile phone. The mobile phone was not just a tool for reporting, but also something that she would use for calls and show it to her kids and family. Few JPHNs were instantly taking photographs of our launch using the mobile phones we provided. I hope I can have many such satisfying moments through my work in health informatics.

But not everything was perfect in all this. I for one realized, that logistics required for a project of this magnitude is not an easy job. Having a team of people in the state and within us at Delhi is a challenge. Discussions of inefficiencies and incapability of the team is a burden in itself to manage. Not everyone may have the goal to change the world and some might even be in it for the money. As for the logistics, we learnt early in the pilot at Himachal Pradesh that a “Step-by-Step” reference manual on using the application was a necessity for the health worker which she can refer to, when she’s at her home and using the application to report. Then there was the purchase of state-specific SIM cards and allocating mobile phone numbers to each health-worker and her reporting office. All this requires lots of planning and managing the logistics and repeated changing of plans has been a tough one for me.

The technical preparation of installing the application, creating the database and installing the server at the primary health centers, block hospitals and district health offices have also been somewhat of a mismanagement. Its been a complex process involving people with different work cultures and behaviors. But in the end the 2 pilots have been successful because its been an opportunity for health workers to report data more efficiently. Final results of the pilot await us, but the start has been great!

Realpolitik of Health Management Information System

HMIS or Health Management Information System has been quite a buzz-word around the Indian health machinery for sometime now. Its just that health-workers are now starting to understand. From the lay-man’s point-of-view, HMIS is a system of managing health related information. It is the process of collecting health-related data, making sense of that data to become information, understanding the relevance of information and making plans on how to use the information.

HMIS being an information system, surely has the scope of being managed through computers and it is done pan-India using an application called DHIS 2 (District Health Information Software v2). DHIS2 is an open-source Java web application and it is free for anyone to use. Free as in Freedom and free as in beer. And this has caused a stir of sorts because there aren’t a lot of things that you get free in life. I am sure some philosopher’s would be quick to tell me that nothing in life is free and so its true with DHIS as well!! But its for fact, cheaper than most other pieces of software competing in the HMIS space.

But we are not here to talk about DHIS, we are here to know the realpolitik of HMIS. Since NRHM (National Rural Health Mission) has poured in money into the health sector, lot of things have changed. It has improved health infrastructure, changed government’s outlook towards public health, generated employment and done a lot more. But it has also brought in realpolitik.

Realpolitik is a theory of politics that focuses on considerations of power, not ideals, morals, or principles. Its “real” instead of “idealistic” and that is sometimes not a bad thing. Politics, in-fact isn’t a bad thing at all!! Health forever has been idealistic, doctors forever revered as people who save lives and government health machinery thought as part of goodwill governance. With these ideals, the Constitution of India enlists “Health as a state matter”. But who has the money and who gives the money, when we are talking about health??

Software systems have enormously increased the reach and speed of information exchange and data collection for health. DHIS2 and pro-HMIS crowd has created a nation-wide awareness on the usefulness of software systems for health monitoring. This has led to a power struggle within different groups not just within the government, but NGOs working in the health sector and consulting groups that work with the states. The power game has just begun with software and technology proving as powerful weapons for whosoever wants to use it. In this hype for HMIS, simple portals that capture data for national level have fetched huge chunks of money and each day additional money is spent on building many more such systems. Then there is the power struggle to capture the market on who trains the cadre of health workers on these new technological advances that has somehow no relevance on health services.

I am just starting to see the “realpolitik”, which is definitely not bad as an ideology... Its just that health always seemed like a noble cause. I am now seeing it as a realist.

Programming: The finest steel has to go through the hottest fire

As the college season is in full flow, I'm back to my philosophical self and I've something for my students... Last week, I was conducting the regular Assembly Programming practicals, where I'm supposed to teach students 8085/8086/8051 ASM programming. Now, as much as I love programming and finding solutions to problem, I also love another thing: Creating problems for others to solve... and even if I know the answers, I would never give away the answer... come what may!!

So we started with all the instructions, available registers and memory allocation. Somewhere down the line, we started with a program to multiply 2 numbers. Now, as simple as I thought it was, my students didn't seem to think the same way. I told them that repeated addition was actually multiplication and they affirmatively nodded their heads, but the program wouldn't just come out from them. After close to an hour and me giving them numerous hints, we were still stuck at the loop!! They demonstrated it easily in a single line of C or C++ or Java, but ASM wasn't just clicking. That's when I affirmed, "Don't worry, programming doesn't come naturally to anyone... We all learn it our own way!!" And the class was stunned, similar to what I've seen at my office's training sessions... and the next question most of the times is, "Then how does it come naturally to you ??"...

"The finest steel has to go through the hottest fire!"... I read it somewhere, but then that's what is true about your individual talents. A singer's voice, a painter's hand, a composer's notes, a poet's verse... all may be claimed to have come from God or by birth, but its the desire to overcome obstacles and practice is what makes them achieve perfection.

Programming is very much an acquired skill, just like most things in life. Even logic for that matter doesn't come by birth. As we observe others follow a pattern of doing things, we follow that same pattern. The more we do the same thing in a way that is acceptable to our intellect, the more it becomes part of our existence and more natural the action becomes. When we first learn to speak, we probably think its harder than anything we have done before. Now as we know to speak, its easier than easy. Same is with programming. The more code that we observe, the more code we write, programming becomes more natural. Whoever says that logic to solve problems came easy, is either boasting his own ego or hasn't thought logically about it.

I was at the vegetable vendor the other day, who was talking of a way to solve the Bihar floods which made some 20million people homeless. His solution sounded really promising and as I know him for some years now, I knew he was intelligent. Huh... I wondered if programming would come naturally to him. He had never seen or used a computer ever but had passed high school. So if he could find a solution, it would mean that programming (read: finding solutions) came naturally to him. So, while he was selling vegetables, I sat beside him and blabbered the same things, I had told to my students at that lecture, but this time it was in Hindi and avoiding most tech-words. While I was speaking, I didn't even know if he was listening... He was busy selling vegetables, with his ~10yr old son and the aunties that came by kept smiling while looking at me. After a while, I asked him if he could multiple 2 numbers, given the instructions I explained... He thought for a while, picked up a few ladies finger (those long green vegetables), which he represented as instructions and explained me a perfect program to multiply numbers, not just 8-bit or 16-bit that would fit in a register-pair, but a multiplication program that would store larger numbers in ordered memory locations... Wow... that was superb!!... Unbelievable and brilliant, I said to myself!! I told him real instructions like ADD r1 r2, MOV r M,  ... and I told him to use those instructions and make a program to calculate modulus of a number. Hmmm... he thought for a while, but this time couldn't find a solution. He asked me many questions though, which he thought would give him an answer... But alas, no solution was found!!

I went back home and realized that may be it is the mumbo-jumbo words that divert our conscious efforts to find a solution. It may very well be the emotion of success that stops us from continuing to find the next problem and its solution. My students had passed earlier classes with 90% and more marks and were among the best students academically in the city.  Did that success get to them ??... Whatever it was, I realized persistence to find an answer to a problem gave us much better understanding of ourselves... May be programming teaches us to understand ourselves better than the problem at hand!!

A Look At Free/OpenSource Cross-Platform Installers

Software Distribution is an essential part of Software Development and can sometimes be the first impression that can make or break the user's opinion about a software. We, as software programmers forget the importance of easy distribution and easy installation of software that we develop. We do not understand the problems that a new computer user or a non-programmer may face. And I experienced this first hand about 2 weeks back, when a physician friend of mine heard that I was working on OpenMRS.

I was lucky enough to work on OpenMRS this summer and learnt a lot more about Medical Informatics during this period than I expected. Hearing this, my friend wanted to install OpenMRS at his clinic which already used Tally (hehe... isn't that innovative??) for storing patient records, observations and prescriptions. He practices at Kolkata, visits different hospitals and sometimes the patients he attended at a hospital come to his clinic. When I told him that OpenMRS was a webapp, he got all excited and I narrated him all the features that OpenMRS could provide and help him manage his patients better through the web, only if he could host OpenMRS from his clinic. I'll skip the other interesting parts and his extra-terrestrial expressions ;-), since we are actually talking about software distribution.

So then came the day when I was about to leave office and he was in his clinic trying to install OpenMRS. It was Independence Day and the clinic was closed but he was excited to experience the new-age medical informatics :-)) When I first got his call he had downloaded the Windows Installer. I was pretty sure it was for an older version and hence told him to instead download the OpenMRS Appliance, which is a VM Image that can be run from one of the virtual machine softwares. Yaw Anokwa made this wonderful Virtual Image with Ubuntu + All Necessary Stuff (tomcat, mysql, demo data) and OpenMRS running. You just have to have VirtualBox or VMPlayer or VmWare Workstation and load the VMimage and wait for Ubuntu to start. It is simple, fast and safe to play with... But for novice users, I just realized it wasn't easy enough. My friend installed VirtualBox and loaded the image. It booted fine, but the network wasn't working and OpenMRS webapp could not be reached from the Windows host. After being on call for close to an hour, we just couldn't make the networking work!! I advised him to install VMPlayer instead and run the image. This time everything ran fine, but some changes had to be made in the Norton 360 Firewall. He kept complaining that Windows XP was punishably slow and then I realized that his 512Mb wasn't enough to virtualize :-( ... So we were back to where it all started!! The Windows Installer that OpenMRS distributes is based on Bitrock. He first tried the OpenMRS 1.1 Installer, but it is an older version that hasn't been upgraded for a year or so... Everything installed fine and he was happy to use it, but it didn't have the features I talked about that were added in newer releases of OpenMRS. I walked him through the manual installation and finally we managed to get OpenMRS up-and-running at 2am in the morning and he having spent about 8hrs on it. Last week when I asked him, he still wasn't using OpenMRS for his clinic and hospital. May be the first experience made him bitter!!

With that episode in my mind, I pledged him that within the next month or so I'll give him and easy to install setup and he'll be happy using OpenMRS. And that's when began my chase to find an easy to use, cross-platform installer framework. OpenMRS has lots of implementations on different platforms (Windows, Linux and Mac) and hence I wanted the installer to be cross-platform. At my office, we generally use Windows Installer or NSIS for making installers. But those are only for Windows. These 2 frameworks are so simple and extensible to use that I was thinking if there was something similar and cross-platform, I could make an OpenMRS Installer in an hour. But sadly, that wasn't the case... I tried a variety of installer frameworks, but couldn't find any of them as simple as NSIS or Windows Installer (msi). The following are the installer frameworks I tried working on:

Installer Framework	Short Description	Problems
1.) Antigen	Antigen (Ant Installer Generator) is a tool to take an Ant build script, combine it with a GUI and wrap it up as an executable jar file. Its primary purpose is to create powerful graphical installers from Ant scripts.	Couldn't get it to execute ant-calls at lots of places. Didn't work in openSuSE 11.0 due to some incomplete ant configurations. Hasn't been updated in a long time
2.) IzPack	IzPack-generated installers require Java. They are simple, efficient and fast to use. Simple executable deployment is best done through IzPack.	Isn't very powerful. Good for simple image deployment, but isn't highly configurable and powerful.
3.) OpenInstaller	A newer cross-platform installer framework that is completely customizable and written in Java. Glassfish uses this installer framework.	Not much documentation. Complex to implement and doesn't look native on all platforms
4.) Netbeans Installer (nbi)	A completely customizable and powerful installer framework. Configuration Logic is written in Java and can be used to do anything and everything that Java programs can do.	Old documentation. Requires some effort to get up and running with all the scripts.

So finally, I decided to work on using the Netbeans Installer. Netbeans Installer already has components like Tomcat, MySQL, Glassfish, OpenESB and their deployment scripts. And I thought it will simplify my effort... Dmitry Lipin of Sun Microsystems, the lead developer of the NBI team has been of great help over the past weeks and has helped a lot in explaining about nbi... While I was building the installer, 2 other colleagues of mine got interested in OpenMRS and have helped build some parts and want to contribute to OpenMRS code in a larger way!!

I have successfully been able to build an Installer/Uninstaller that can deploy Tomcat/Glassfish, MySQL and the OpenMRS web application on Windows, Linux, OSX, Solaris. The demo data set, JRE/JDK and starting the respective servers are yet to be completed.

Update: The OpenMRS Windows Installer based on Bitrock has been upgraded to install the latest version of OpenMRS. Is it useful for the OpenMRS community to have a cross-platform installer?? Or do the Windows guys only need an Installer ??

Practice Download Day: Firefox 3 RC2 Released

Firefox 3 download is coming closer and Mozilla has just released the latest version of Firefox 3 RC2. The version is faster, smoother and uses lesser memory compared to previous versions. Firefox RC2 is the last one before the final release and is a good time to see if all the bugs are ironed out.

Probably another important thing is to test the server capacity for the download day. Today's release may be useful for the download. Download it from here and Have Fun!!

Google Already Does What Wikia's Trying - Social Search

It wasn't all that long back when everyone was disappointed with Wikia's launch of alpha search. With Wikia Search everyone was expecting it be a Google killer from day one and the disappointment was made very public by a lot of tech bloggers. At that time, I thought that it was a little harsh of everyone to be criticizing something so "alpha". But I guess the Web is now v2.0, its growing rather rapidly and people can't wait for social evolutions any more.

Give Social Search Some Time

Wikia Search is about social searching where people search for a specific keyword, follow links, vote for good search results and the contribute a small relevant mini-article. Darwin realized it long back that evolution is slow and made the world know about it. Social evolutions in behaviour patterns are even slower... Then why in the world would anyone think social search will start working from day one?? The success of social networking in the last 2-3 years may be the primary reason where people have started to expect faster growth from social phenomena on the web. But the reason for success was that people did not change their behaviour. They just got a completely new experience since social networking didn't exist earlier.

Search on the other hand already exists and people are behaviorally already accustomed to using algorithmic search. Better web users can already think of good keywords and get their results google'd (since that's a verb these days). So really, Wikia or for that matter social search will take its own time to grow. And if I were to predict, after it matures it'll be better than algorithmic search.

Google understands Social Search

I performed 3 tests over a period of a 30 days and have come to the conclusion that Google performs Social Search, but seems to be a little confused on how much priority it needs to give to social valuations.

Test 1 (Google Reader & Feedburner): I created 6 blogs with the same content and kept posting and pinging the Google BlogSearch equally. Each blog was updated daily at the same time from the same location. But I subscribed different users to the feeds differently using feedburner and google reader. Initially without any feed subscribers, all the blogs would randomly appear above/below each other for a week. After a week, with different number of subscribers, I found that Google would display that blog first which has more number of feed subscribers. Thus Google very well knows which blogs have more subscribers and displays those in the result higher than the others.

Test 2 (Search Result Clicking): This was a weird one to detect. I found that sites which are clicked more in the results got the better ranking in the future. But this was not a general higher ranking in the search results. I observed that it was only limited to that IP subnet where the clicks were made. It meant that Google was able to understand which sites the user visits and hence brings it above others in the search results.

Test 3 (Orkut & GTalk Contacts): Google's social networking site Orkut was my next target. Considering orkut scraps and communities weren't part of Google's index, I didnt think they would have any effect, but I was surprised!! I couldn't spam a lot of friends with the blog links, but I did some in communities which had foolish admin's accepting me as a member. I spammed about 200 communities with the blog links and found a change in the search results. These weren't as straightforward and sometimes the higher ranking would be lost and would come back again at times. But there were more times when pages with orkut links ranked higher.

When I did similar spams with Gtalk contact though none of sites changed. They were still random in the results.

Random Observations:

• Blogger/Blogspot sites ranked higher: Initially I didn't realize this, but later found out about 2 weeks into the testing that Google was ranking blogger/blogspot sites higher in the results. It was surprising at first, but then I found out that Blogspot's subdomain had a higher ranking. Other blogs that are subdomained to good domains also had a similar effect. The conclusion I reached is that Google seems to be checking the domain name first and then its content. Even if the subdomain name was more apt to the search keyword, Google cared more about the domain name.
• Knol could be the missing link: I have not tested Knol yet, but it could be the primary decision maker for the Google's engine on social search. Knol articles and the links it mentions could be more valuable because Google has stated that Knol articles are expected to come from experts on any given subject.
• Gmail could also be used: I tried sending lots of emails (16000 in all) with blog links. None of it had an effect :( I think Google could use it very effectively to improve search results. But I guess there could be some privacy concerns with that happening, although Google already has the rights to index our emails in the EULA for Gmail.

The tests are completely my own observations and are limited in resources (6 blogs) and time (30 days). But they give us a slight indication that Google may be social searching already. How much they rely on it is completely unknown and will be unknown forever... Google already knows so much about society on the Internet that they are probably the only company (with enough resources) that can accelerate social search and implement it successfully today!!

Windows Server 2008 WDK Released

Today, Microsoft released the RTM of the Windows Driver Kit (WDK) to developers and beta testers and is available for download on the Microsoft Connect website. The Windows Server 2008 RTM WDK is used to create drivers for Server 2008 as well as earlier operating systems like Windows 2000, Windows XP, Windows Server 2003, Windows Vista. Microsoft advises everyone that this is the supported platform for the development and release of all production drivers.

WDK was earlier known as Windows Driver Development Kit (DDK). With the release of Windows Vista last year, Windows Driver Foundation (WDF), Installable File System Kit (IFS) were added and a separate Windows Logo Kit (WLK) was also created for signing the drivers.

A few highlights in this release are:

KMDF 1.7:

• A new wait/wake policy for parent drivers
• New samples
• New Tools

UMDF 1.7:

• Now supports Windows Server 2003 along with Windows Server 2008, Windows Vista, and Windows XP
• New debugger extensions
• New and updated samples

WDF 1.7:

• WDF 1.7 is native on Windows Server 2008 and Windows Vista SP1
• WDF 1.7 is completely backward compatible with WDF versions 1.0, 1.1, and 1.5

WDTF:

• SDEL has been added to WDTF to make querying simpler and easier

SDV:

• SDV supports KMDF drivers in addition to WDM drivers and parallelization over rules. Now, rules will be verified in parallel.

PFD:

• PFD is enhanced with driver-specific annotations that make it an even more powerful tool.

DSF:

• USB Power Management and now more robust and stable

Docs:

• New look of the WDK documentation for Server 2008/Vista SP1
• new docs for Viridian and Kernel Transaction Manager
• Subset Filtering (allows you to search by technology subset such as Audio)
• Requirements Block information for all new topics
• Kernel Reference technical scrub and TOC restructuring

After I downloaded, the new documentation felt simpler for navigation and allows easier understanding of the process of creating drivers. This release will be useful for all the hardware vendors who will be creating the next version of the drivers using this RTM release. I'm currently trying to find some mention on creating friendlier drivers supporting virtualization for Microsoft's new Hyper-V, but its surely not in the bold-print and would be useful if someone can point it out to me!!

SQL Server 2008 Will Be Late

Microsoft was going to release its database management system, SQL Server 2008 in Q2 of this year, but now has mentioned that the product will require some more time to be completely ready for release. Thus, the delay currently means that Microsoft will be releasing SQL Server 2008 sometime in Q3, 2008. Even this date will depend on how fast Microsoft can work on the product and get it ready for release.

Every major Microsoft product has been delayed off-late. A few examples include Windows Vista, Home Server, Sharepoint, Office for Mac, Visual Studio 2008. This either means that Microsoft has been doing a lazy rest or may be Microsoft has been promising too much to early. It could also be the case where people are expecting product releases like before, but Microsoft is spending more time than before in refining and polishing the products. Either ways, its not a good sign for the world's largest software company.

The delay was part of a roadmap clarification, another way to say that we screwed sometime in the planning phase. But all the delay isn't affecting Microsoft much, since its still able to make good profits and is able to sell enough products. SQL Server 2005 which was delayed by some 18 months was well received and is hosting a lot of databases at the moment.

The clarification on the site reads like this:

To continue in this spirit of open communication, we want to provide clarification on the roadmap for SQL Server 2008. Over the coming months, customers and partners can look forward to significant product milestones for SQL Server.  Microsoft is excited to deliver a feature complete CTP during the Heroes Happen Here launch wave and a release candidate (RC) in Q2 calendar year 2008, with final Release to manufacturing (RTM) of SQL Server 2008 expected in Q3. Our goal is to deliver the highest quality product possible and we simply want to use the time to meet the high bar that you, our customers, expect.

This does not in any way change our plans for the February 27 launch and we look forward to seeing many of you in Los Angeles and other events around the world. Please keep the great feedback coming and thank you again for your ongoing support of SQL Server!

Visual Studio 2008 To RTM On 19th Nov

Microsoft has finished all of its work on the highly anticipated Visual Studio 2008 and will be Released-To-Manufacturing (RTM) on 19th Nov, 2007, which is tomorrow. Visual Studio 2008 is the most popular development platform for Windows and Microsoft has made this version specifically for development of Windows Vista. Visual Studio 2008 comes bundled with .NET framework 3.5 which I covered earlier. Visual Studio 2008 comes after 3 years and developers are excited for LINQ, XAML Visual Designer and few other new features that are part of the .NET Framework.

The MSDN Subscription Blog notes that it'll be available for download to MSDN subscribers very soon and I expect it to be sometime next week soon after the RTM. Earlier, on Nov 7th, Microsoft had announced that the Team System of Visual Studio 2008 will be shipped at the end of November. The Team System is the largest of the Visual Studio Suite and is specifically designed for Team Collaboration in mind.

Jeff Beehler, Team System Chief Of Staff, said: "We're very close to shipping Team System 2008 and expect to make it available by the end of this month. This will include all of the team editions (Development, Database, Test, Architect, Suite) as well as the Load Test Agent and of course Team Foundation Server."

Netbeans 6.0 - The IDE to own

I have just qualified to the NetBeans IDE 6.0 Community Acceptance Testing program (NetCAT) and before I begin giving some feedback, I thought... how about sharing some info about the NetBeans 6.0. What it is all about and how it has improved and how it can become the IDE for all programming languages.

Any programmer will tell you how much he/she loves the IDE. How much time of their lives they spend looking at the IDE screen? and how much the IDE is part of their creative exploration (programming)? These days with so many programming languages, a developer in my organization comes across a minimum 3 IDE or WYSIWYG Editors. We use Zend, netBeans, Visual Studio 6, VS2005, Dreamweaver, Eclipse, MS Expression Web, Anjuta, KDevelop ... and even after that I'm sure I missed something that we use. You see what I mean when I say there are zillions of IDEs.

Since, I have been programming a lot in Java, netBeans has been with me since version 4 and I always thought it had promise. Initially though eclipse was better! But I believe netBeans 5.5 was awesome and 6.0 is gonna beat a lot of other IDEs out there. Im currently doing a short preview of netBeans 6.0 Vs Visual Studio 2008 (my next post), both IDEs from different worlds, but then they still are IDEs right!! I know its apples vs oranges, but then netBeans 6.0 is not just a Java IDE, its got a lot more to it... Its a platform as well but I'm looking at it as an IDE for many languages. The templates to build Ruby on Rails blogs is just an example how much netBeans 6.0 team is trying to make it an all-language IDE.

The following are some of the new features to netBeans 6.0:

1.) Improvements to the Code Editor:
a.) Smarter & Quicker Code Completion - Faster and better options & methods shown.
b.) Highlighting - common variables are highlighted

2.) Language Support:
a.) JRuby/Ruby/Ruby on Rails Support - Excellent template, auto-structuring of folders. Great Ruby Debugger. Nice RHTML file editing and highlighting. Example Video
b.) Improved Javascript and CSS Editing - Although dreamweaver is better, its gotten better than the previous versions.
c.) Schielmann language support - Its a technology that allows anyone to add support to new languages. Php and Jython plugins are available.
d.) C/C++ Support: Greatly Improved support for programming in C/C++. Libraries are easily accessed and also has pretty good code editing.

3.) Swing GUI:
a.) Swing database applications - Probably the single-most awesome feature from netBeans 6.0. Great for making RAD for people who want to make fast database application.
b.) Beans binding: You can now bind beans and control their behavior from the GUI. You don't need the BDK (Beans Developement Kit) as a GUI for looking at beans behavior and property editing.
c.) Swing Application Framework (JSR 296): Awesome support of application state saving. The state of the application is saved as default. You don't have to worry about the size of the window of your application. It'll come up at the same size that you closed last time, and for all this you dont have to write a single line of code. netBeans does it for you.

4.) Profiling:
a.) Integrated Profiler - The profiling tool within netBeans is elegant and perfect. May be you can compare it to LaodRunner yet, but then it does most things perfectly.

5.) Web & Java EE:
a.) Improved Visual Web (JSF): After Sun decided that Studio Creator will be netBeans, you find that there's no better tool for making JSF web applications that netBeans 6.0. Just drag-drop the components, write some business logic as EJBs and you have a well made web app.
b.) Support for AJAX-enabled JSF components
c.) Great Javascript editing, with error and debugging, not found in other WYSISWYG editors. CSS support is improved.
d.) Brilliant flow management for Web application. You can now make nice design for page flows in netBeans 6.0

6.) Java Mobility Support:
a.) No other J2ME/Java ME IDE has the ease of use and quick drag-drop MIDlet developement like netBeans. Fast and feature rich. You can make flow diagrams with it. Designing mobile apps were never easier.
b.) Support for APIs: JSR-172 stub compiler, JSR-226 SVG support is worth a wow!!
c.) Game Builder: Easy tiling allows nice design of game screens and sprites

7.) SOA:
a.) Graphical WSDL editor: Don't we all love interoperable WSDL documents?? But hate the binding. netBeans has great visual tool to do all the binding work for you.
b.) Easily edit the transformation on XSLT editor
c.) Improved interoperability with .Net through WSIT

8.) UML:
a.) Enhanced code generation: Lesser difference in your designs and code. I love it!!
b.) Don't we architects love design diagrams. Making them from the IDE, is just a boon. Brilliant UML editing improvements, although some standalone tools do a better job, its still a great thing to have in your editor

9.) Misc:
a.) Improved graphing in the platform API
b.) A new lexer in the netBeans seems to make it faster for code editing

There's a lot more to netBeans 6.0... You can read about all the improvements here

See you on my next post, when I compare the upcoming IDEs Visual Studio 2008 and netBeans 6.0, with references to few other IDEs