Monday, December 21, 2009

Saving a streaming video / presentation

Ever wondered to to save those streaming videos from Youtube, Metacafe, InfoQ, and so many others? I always wanted to save those streams, so that I could watch them again and again even without an Internet connection.

I am a faithful Firefox user, and one of its wonderful plugin - DownloadHelper has helped me for a long time to capture and save Youtube videos as flash video files on my local hard disk. That was quite helpful. But I always wanted to save those InfoQ videos as well, but unfortunately, DownloadHelper did not provide support for InfoQ (among others).

It was only recently, that I used the conversion feature of DownloadHelper, in which I was converting one of those Youtube videos to be converted to wmv format. I noticed, that the conversion started only after the whole of the video stream was played on my browser. And I could readily guess, that the conversion must be using one of the libraries on my machine. So I looked at the process to find that it was using ffmpeg command to do this job. I also noticed, that actually every video was buffered to the /tmp on my Ubuntu. The filename I could see was something like FlashVOZFnS.

This was a great news, because this was applicable even to InfoQ videos. Naturally, if a movie plays on your machine, it must be stored somewhere before being played on your browser. So now if you want to save your next streaming video, just identify where in your system does the flash player saves its movie temporarily before being played on your browser, and save that file to your videos folder, and possibly give it a .flv extension too.

Note
  1. As soon as you close the tab containing the video, the temporary file is discarded.
  2. Not all videos may be fully buffered before playing. Very long videos and webcasts may not be fully buffered.

Converting flash movies to other format (Linux)

And now if you also want to convert it into other formats, its very easy in Linux with the following commands:
FLV to WMV: 
ffmpeg -i /home/thomas/Desktop/infoq.flv -y -v 0 -acodec wmav1 -b 1000kbps -f asf -vcodec wmv1 /home/thomas/Desktop/infoq.wmv

FLV to MPG:
ffmpeg -i /home/thomas/dwhelper/Desktop/infoq.flv -y -v 0 -f asf -vcodec mpeg4 /home/thomas/Desktop/infoq.mpg

There are so many other options with this command, even up to video capturing your screen, but I am not very expert at those. So I leave it to you to experiment with those options.

Update

I could notice that actually DownloadHelper can download InfoQ videos, but just that you should have only this one tab containing any video to download. In other words, if you have multiple tabs with video contents, DownloadHelper seems to get confused, so you may not actually get a download link help. Also, getting the Slides on InfoQ is also easy. If you are a good developer, just see the source code of the page, and you will understand. Right now I see those videos and slides at my ease stored on my hard disk.

Tuesday, December 15, 2009

Configuring log4j with XML in OSGi

Sometime back, I had to make perf4j work on OSGi. It required log4j as the logging library and purely demanded log4j.xml (it would not take the .properties file since the configuration must attach downstream appenders). I assumed that it should be simple, but unfortunately it was not. In the rest of this post, I describe how I could get around this, and used log4j.xml to configure log4j logging in OSGi.

Wednesday, November 18, 2009

Microsoft ASP.NET MVC - Why Java web developers should reconsider action frameworks?

Microsoft's ASP.NET MVC is a paradigm shift in web development from its pioneering component based development offered with WebForms to action based web framework. This was a natural evolution due to the pains in the component based web development. While the Java community started out with the action based frameworks, the shift of the community seems to be on a higher side for the component based web development led by JavaServer Faces (JSF) in the recent years. The shift of Microsoft towards action frameworks should make the Java web developer's heads again turn towards action frameworks and reconsider their choice towards the paradigm for web development. Perhaps a thinking about polyglot web development is needed, rather than abandoning action frameworks.



Thursday, October 29, 2009

Evolving persistence, SQL and ORMs to the next generation

Recently there has been a lot of discussion on the web describing the ORMs as a thing of the past, and its associated discussions that agreed and disagreed with their arguments presented to various degrees. The discussions even took a long conversations across the twitter space too.

Persistence models and application evolution

Persistence is important for any application. Over the years this has been done in various ways. While RDBMS and SQL were the most obvious choice till few years from now, recent times have also seen successful practical usage of other persistence models which include Amazon's SimpleDB, and Google's BigTable among others. These non-relational alternatives fall under the NoSQL (or “post-relational”) persistence models.

Naturally it becomes quite obvious that over the time, relational databases have proven inadequate to satisfy the growing needs of the Web2.0 applications. The advent of the new-generation non-relational databases are a proof to this. Every persistence model has a niche area to which they cater. The newer databases have has also made people to think about "mix-n-match" use of the various persistence models as people talk more and more about polyglot persistence.

The relation-object impedance mismatch

Since RDBMS has a solid theoretical foundation, and has been used across various applications, its existence for another century cannot be ruled out. Yet new advances in persistence like CouchDB, BigTable, SimpleDB, Graph databases, etc are most welcome steps in advancing the persistence mechanism.

Even with newer persistence models, a lot of the enterprise applications still use, and will continue to use relational databases. Applications that use up this data are largely represented as objects. While object model seem to be the most obvious choice for application development, the mismatch between the relational model and the object model comes as the reason for a lot of hues and cries that have given rise to ORMs. The learning curve and the pain in using these ORMs have now given rise to debates questioning the existence of ORMs in the future.

More than debating over the existence of ORMs itself, rather a deep look is needed, to the problem which the ORM is trying to solve. Why does the ORM exist? The most popular answer would be impedance mismatch. But the mismatch is pointed out between relational data representation versus the object model.

Overhauling the persistence mechanisms

Often the data storage mechanism and its interfacing API are seen as an integral part, including the RDBMS data storage and SQL. They are often seen as one, but are actually two separate components of the modern day relational databases. They were created at a time when the existence and usage of OO was questionable. Even now when they are used for a long time, it can be argued that SQL and RDBMS themselves have a mismatch in the paradigms. Now when OO has gained so much importance, it is time that the RDBMS and SQL should be re-looked from the OO perspective.

We can consider two aspects in any persistence mechanism:
  1. The way data is stored (relational, key-value, hierarchical , column, JSON etc) using various data structures and storage mechanisms. Closely associated to this data storage will be the related methods of storing and fetching the stored data (algorithms used to fetch data based on persistent store itself - eg relational algebra) that could be probably exposed through function calls. The function calls will be closely associated to the data store mechanism itself.
  2. Interfacing provided to interact with persistent store (SQL, NoSQL, ObjectQL, programming language API etc).
Often these two aspects are baked together (in our minds most of the time). This thinking should evolve and these two should essentially be seen as separate, and ideally pluggable. We talk about adapter pattern in programming languages, but the same concept could probably be applied to persistence mechanisms as well. The persistent storage and its storing and fetching mechanism should allow widely different interfacing provided to interact with it. DB vendors can provide few interfacing mechanism, and also allow others to write other interfacing mechanism that should be pluggable.

Interfacing will expose the persistent store in various forms - relational model, or object model or even other models used now and may come in future. Mapping of the datastore persistence model to the exposed model should reside with the interfacing mechanism used.

Concluding thoughts

Achieving this practically may not be very easy, but not impossible. Once this is achieved, ORMs will rather shed off the mapping weight and evolve as frameworks that enhance developer productivity on issues like caching, application scaling etc. Rather, ORMs will no longer be ORMs and probably a new breed of softwares will evolve that will take care of the cross-cutting concerns for an application when dealing with persistent stores.

This may look crap to some, but then that is what I wish. I am an application developer presenting my ideas on what a database itself can provide, and how applications can leverage the new capabilities. The issue needs to be looked at by db vendors and research scholars. DB vendors and db developers need to give deeper thoughts to the problems faced by application developers in their need to persist application data. How many of us use the bare databases (using only SQLs probably) without an application? The current forms of persistence may be right in their own theoretical framework, but it should evolve to fit in seamlessly with the application development space, where it is used.

And till that happens, I will have to carefully weigh my choices among NoSQL, SQL and ORMs, and muddle with the problems that have already heated up a lot of discussions in the application development space.

Tuesday, October 27, 2009

My state of affairs - part 2 (looking for new projects)

In my previous blog post, I mentioned about the my state of affairs, of how things have been moving in last 2-3 years, and why I was so irregular at posting blogs. The state of affairs was actually not finished yet. I continue it here.

Looking out for new job/consulting opportunities

My current employer is finding it difficult to sustain its employees. While some employees quit at the start of the turbulent times, some of us including me decided to stay strong with the company. And now more than one year after showing signs of trouble, the company is not yet able to fully recover and get back to normal business. The management is so kind to ask us to look out for other jobs as an open offer to us. We can take our own time to move out. Although the management has plans to bring it up again back to business, it doesn't want its employees to be dragged across the difficult times any longer. As our work is tapering off, I will be looking out to see if some good offers can come along.

So if you are looking out for an experienced developer or consultant, with proven track record in providing effective business solutions with good exposure to various technologies, try me - I will be happy to help you out. I specialize in Java/EE, Spring, Maven and OSGi (thats about building truly modular Java applications). And if you want few more experience hands, may be my company can help you in your IT endeavors, specializing in creating robust modular and scalable Java applications.

In the meanwhile I will be working on some OSS projects and writing blogs. So, keep an eye on this blog. My blog posts will give a good understanding of various technologies and how the technology can be best put to use in solving real-life business problems.

I am also working out on a project (of my own) that is a good mix of some technologies I have already learned, or have scratched the surface. It will provide the ground for me to build depth of my technical knowledge as well as serve as a ground to build up applications for my client that require ground-up work. Some time down the line, I will also give out this code to the OSS space.

If you would like to know more about me, follow the links below: