Showing posts with label Programming. Show all posts
Showing posts with label Programming. Show all posts

Getting Started with React Native

React Native is a framework that allows us to create Native Android and iOS applications using JavaScripts by Facebook. It is a extention of React JavaScript framework available for Web application development. With React-Native we can create native android or iOS applications without knowing much about native application development. It opens up the native application development to web developers who knows JavaScript and CSS.

Read more at my new blog >>
 

Easy way to Mock REST services

 

Caution with Java Autoboxing

 

Simple Introduction to Scalaz

Scalaz is a great library to make your Scala code more compact and to reduce some boilerplates. But getting started with it requires considerable effort. With its large collection of different operator and very functional nature most people are afraid to use it. Specially for people like me who are with more imperative programming background, it is not easy to get familiar with scalaz.

Recently InfoQ published a presentation named "Scalaz for Rest of us" by Adam Rosien, which introduces some basic building blocks of scalaz. Explanations are really simple and will help you to shed your fears on scalaz.

 

Stubbing Asynchronous Http Services using Wiremock

Wiremock is simple library which can be used to stub Http Services. It is a handy tool when you want to write unit tests against some Http service(eg: Http REST web service). It has direct support for JUnit using @Rule feature. We can write unit tests to stub different http requests and verify those requests and responses very easily.

One weakness of wiremock was it only supported synchronous  http request mocking(i.e: You send the request and stubbed wiremock server send the response). But I wanted to mock Asynchronous REST server. In asynchronous scenario in addition direct http response, server is initiating another call back request to the server. This method is commonly used by web service operations which take lots of time to process. Flow of such operation would look like follows.

When a client want to get some operation done;
  1) Client sends the request to the server
  2) Server accept that request and give that accepted response as direct http response
  3) Server do the processing and send the actual result as a new http request to the client

This feature is implemented in my wiremock forked repository. Still this is not being merged to main wiremock repository. So if you are interested in this you'll have to clone my forked repo, build it locally and publish the jar file to your local maven repository.

Stubbing asynchronous request is easy.


In most asynchronous http services, some kind of Id is being used to map original request with the asynchronous response. There can be two ways of doing this.
  1) Server generates transactionId and sends it with the immediate response
  2) Client sends the transactionId and server echo that when it sends back asynchronous response

If you want to use the second method with wiremock, you can use setEchoField method.

At the moment there is a one limitation with this. You can only use this facility with JSON requests and responses.

You can find the complete test class at github.
 

Exclude resource files from jar file created with SBT

I use SBT to build my scala projects. In addition to scala source files I have some resource files in my project. Those are some configuration files and localization resource files.

When I build the jar file using SBT package command all of those resource files also get included in the jar file. When those resource files are included inside the jar, modifying them becomes difficult. I have to extract the jar file, edit them and create the jar file again.

Ideal way of handling resource files are not to include them in the jar file, rather keep them separately and include them in the class path when running the application. How to exclude them from SBT package task was the problem I had. After some googling I found out that this can be achieved by including mapping task in the build. Following is sample build file which uses mapping to exclude various types of resource files.


val excludeFileRegx = """(.*?)\.(properties|props|conf|dsl|txt|xml)$""".r

  lazy val myapp = Project(id = "myapp", base = file("myapp"),
    settings = baseSettings ++ Seq(
      name := "My App",
      mappings in (Compile, packageBin) ~= { (ms: Seq[(File, String)]) =>
        ms filter {
          case (file, toPath =>{
            val shouldExclude = excludeFileRegx.pattern.matcher(file.getName).matches
           // println("===========" + file + "  " + shouldExclude)
            !shouldExclude
          }
        }
      },
        libraryDependencies ++= Seq (dispatch),
      libraryDependencies ++= testDependencies))

 

Creating Java Command Line Applications using jLine

Most common way of writing a command line application in java is to use BufferedReader to read user input. But if you want more features like tab completion, you have to write code to handle it from the scratch in BufferedReader method.

There is a nice library called jLine http://jline.sourceforge.net/ which can be used to write nice CLI apps without much effort. It has out of the box support for;

  • Command History 
  • Tab completion 
  • Line editing 
  • Custom Key Bindings 
  • Character masking
I've written simple sample application on how to use jLine. You can check that out in github https://github.com/sandarenu/sample-apps/tree/master/java-cli-app
 

How to Use Multiple GIT Accounts

Yesterday at HMS we moved some of our projects to GIT from SVN repository. So in order to access that office GIT repository I had to create a new ssh key using my office email address. I already had a ssh key which I used to access my GitHub account. So once I created new key I was unable to access GitHub since in normal configuration ssh can only have one public key.
But I wanted to find a way which I can use both office GIT server login and my GitHub login. After doing some searching on google I found a solution to that. For that we'll have to create ssh config file. So I created following config file at my .ssh folder
vi ~/.ssh/config

# hms-git
Host hms-git
  HostName git.myoffice.com
  User git
  IdentityFile /home/sandarenu/.ssh/id_rsa

# github
Host github.com
  HostName github.com
  User git
  IdentityFile /home/sandarenu/.ssh/id_rsa_github


Once this config file is saved I could clone office git projects using
  git clone git@hms-git:myproject

To clone GitHub project I could use
  git clone git@github.com:sandarenu/task_tracker.git



 

Using SoapUI for Web Service Testing

SoapUI is a great tool which can be used to test web services. In this post I'll show you how to set up SoapUI project for WS testing.

Create new project from File ->New SoapUI Project, and in "New Project Dialog" give name and the location of the WSDL file.


You can find the sample wsdl file I used for testing in github https://github.com/sandarenu/soapui-test.

Web Service Mocking

Service mocking is required when you are writing a web service clients and either you don't have the web service implemented or you don't have the access to the real service. Here's how to do that using SoapUI.

Create Mock of the web service by right clicking on the project and selecting 'Generate MockService'.

Once mock service is created you can mock the operations by clicking on the small icon in the service mock dialog.




Double click on the operation you just mocked and you'll get a dialog to configure mock responses.
You can configure any number of mock responses and then select how those response to be dispatched when our mock service receives WS request. There are many dispatching strategies available such as "Script", "Sequence", "Query Match", etc...

When you do some integration testing you need your mock service to respond based on different parameters of the requests you send. In such cases you can either use "Script" or "Query Match" dispatch mechanism.

Web Service Testing
We can use SoapUI to create test cases to test a Web Service we developed. To create a test suite right click on the project and select "New TestSuite" from the context menu. Then right click on the test suite we just created and select "New TestCase".



Once we have created the test case we can add test steps to it. Lets add a test step to test time for time zone 'lk'. Right click on the test steps dialog and select "Test Request" and create the request you want to test.
Once test request is created we can add assertion step to it. Click the "Assertions" button at the bottom of the test request editor.


Right click on the assertion editor and add "xPath Match"

 Put //Time as "xQuery Expression" and put value you are expecting as "Expected Result"

Now you can test this request using the Mock WS you created before. Start the mock service by clicking on the small green arrow found in the top of the "TimeServiceMock". Then open the test case you created above by double clicking it. Then run the test case again by clicking small green arrow in the to of the dialog box.


If every thing went well you'll be able get success result for your test case.

This is just the start. You can then add more test cases and assertions and improve your test suite. SoapUI site has good documentation on how to do functional testing. http://www.soapui.org/Getting-Started/functional-testing.html
You can refer that and get more information.

You can also find the SoapUI project for this example at https://github.com/sandarenu/soapui-test. Download it and open it using "File -> Import Project"

Happy Testing.....
 

MySql - Ms Sql Server Database Portability

Few weeks back I was given a task to migrate a Java application using MySQL as its database to MS SQL Server. It was actually not a full migration, rather application should be able work with both MySQL and MS SQL Server. Since we had used Hibernate in our application I thought it will be a easy thing. But when I dig deep in to the application and started migration I was proved to be wrong. I encounter lot of issues and hence thought of documenting them so any one can get some help. Major headache was the difference in some direct SQL commands used in the application. Following are the list of issues I encounter and how I solved them.


1. JDBC Driver Issue with the Microsoft JDBC Driver
    First thought of using JDBC driver provided by Microsoft for MS SQL Server 2005.  But when I ran the test cases I got few issues with it.
    1). Gave execption "org.hibernate.MappingException: No Dialect mapping for JDBC type: -9 "
        After some investigation found out that it was due to NVARCHAR datatype. There is another opensource driver called jtds. Which is more actively developed as well. With that driver I did not get that MappingException.

2. Data type differences
    For queries like "SELECT count(id) From myTable", MySql JDBC driver returns datatype java.math.BigInteger and with MsSQL jtds JDBC driver we get datatype java.lang.Integer. So I had to convert it to string first and then convert to long
    String countStr = summaryRow[1] == null? "0": String.valueOf(summaryRow[1]);
    long count = Long.parseLong(countStr);

3. Nullable Unique columns not supported in MsSql Server. In Sql Server NULL is taken as a value and hence can't have multiple rows with null, but in MySql we can have Nullable unique columns. So when table schema is generated through Hibernate we have to execute additional update query to perform a workaround for this issue. Here we add new column called 'my-table_id_add' to the table.

declare unique_key_list_my-table cursor for
    select OBJECT_NAME(OBJECT_ID)
    FROM sys.objects
    WHERE type_desc LIKE '%CONSTRAINT' and OBJECT_NAME(OBJECT_ID) LIKE 'UQ__my-table%'

OPEN unique_key_list_my-table
    FETCH NEXT FROM unique_key_list_my-table INTO @keyName
    set @RowNum = 0
    WHILE @@FETCH_STATUS = 0
    BEGIN
      set @RowNum = @RowNum + 1
      print cast(@RowNum as char(1)) + ' ' + @keyName
      if not @keyName is null
      begin
           select @sql = 'ALTER TABLE [my-table] DROP CONSTRAINT [' + @keyName + ']'
           execute sp_executesql @sql
      end
        FETCH NEXT FROM unique_key_list_my-table INTO @keyName
    END
CLOSE unique_key_list_my-table
DEALLOCATE unique_key_list_my-table

ALTER TABLE my-table ADD my-table_id_add AS (CASE WHEN my-table_id IS NULL THEN CAST(id AS VARCHAR(30)) ELSE my-table_id END);
ALTER TABLE my-table ADD CONSTRAINT UQ__my-table_my-table_id UNIQUE(my-table_id_add);


4. TIMESTAMP in MS Sql server is not functionally same as MySQL. Have to use GETDATE() function for that.
    e.g: Create table timestamp_test DATETIME default(GETDATE())

5. In Ms Sql Server we can't use 'user' as table/field name, but that is possible in MySql.

6. LIMIT is not supported in Ms Sql Server and there is no easy way if we want go select some range of data. In MySql we use "SELECT * FROM my-table LIMIT 10, 20" to select records from 10 to 20, but in Ms Sql Server we have to use something like "SELECT * FROM ( SELECT *, ROW_NUMBER() OVER (ORDER BY id) as row FROM my-table ) a WHERE row > 10 and row <= 20"
 

Integrating YUI Compressor to your Maven Web Project

Size of the javascript files and css files in a website has a big affect on its performance. Especially in slow connections lot of time will be spent on downloading those files. We recently came across such situation. Our website was using jQuery and some other javascipt libraries. Some of them were more than 100kb. Sometimes it took more than 10 seconds to load a page. When we view the loading statistics using developer-tools in Chrome it showed that browser had downloaded more than 500kb of scripts and css.

One way to reduce this script size is to compress them. Our script files contained lot of comments, licencing statements, nice formattings, long meaningful variable names etc... which increases the file size. YUI Compressor is a free utility by Yahoo which can be use to compress our scripts by removing above mentioned things. It is jar file and you can use that and compress file by file.

But that is not very practicle to keep those minimized scripts in develepment since after minimizing you won't be able read them and edit them when needed. Best method is to do the minization when you create the war file. Luckly there is a maven plugin for this. Following is the plugin component you can add to your maven pom.


<plugins>
....
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<configuration>
<webResources>
<resource>
<directory>
${project.build.directory}/minimized
</directory>
<targetPath>/</targetPath>
<filtering>false</filtering>
</resource>
</webResources>
</configuration>
</plugin>
<plugin>
<groupId>net.sf.alchim</groupId>
<artifactId>yuicompressor-maven-plugin</artifactId>
<version>0.7.1</version>
<executions>
<execution>
<goals>
<goal>compress</goal>
</goals>
</execution>
</executions>
<configuration>
<webappDirectory>
${project.build.directory}/minimized
</webappDirectory>
<nosuffix>true</nosuffix>
</configuration>
</plugin>
....
</plugins>



When you run mnv clean package YUI compressor will compress the scripts and put them in target/minimized folder and those will be used to create the war file.
 

Reducing the Time Between SCTP Message Received and SCTP_SACK Sent

Last few weeks I was involved with a development related to SCTP protocol in Java. I have also done two posts on how to get started with SCTP. During that project I came across a strange issue. That is I receive two responses for all the packets I send to the peer. After doing some investigation we found that it is due to that from my application SCTP SACK message is sent after 200ms. Because of that delay peer thinks that I have not received the message and sent it again.
That 200ms is a configuration based on the SCTP RFC. According to that an acknowledgment(SACK) should be generated for at least every second packet received, and SHOULD be generated within 200 ms of the arrival of any unacknowledged DATA chunk.So to fix this issue I had to find a way to reduce the delay between message receiving and SACK sending.
We can reduce that time delay by editing the value of /proc/sys/net/sctp/sack_timeout property file. Just use following to set timeout to 50ms. 
echo 50 > /proc/sys/net/sctp/sack_timeout
After doing that that two response issue was fixed.  Thanks Chris at sctp-dev@openjdk.java.net for helping me on fixing this issue.


 

Catching JVM Shutdown event

Shutdown hook is a mechanism provided by java for us to handle shutdown event of our applications. We can attach any piece of code to this and execute them just before JVM exits.

public class ShutDownHookTest {
public static void main(String [] args) throws Exception {
System.out.println("System Started...");

Runtime.getRuntime().addShutdownHook(new Thread( new Runnable() {
public void run() {
System.out.println("System is Shutting Down...");
}
}));

Thread.sleep(10000);
}
}


Compile and run this sample code and press Crt+C to exit the application, you can see that "System is Shutting Down" message prints before application exiting.

But we have to be careful when using this, because if the shutdown code we attach to the shutdown hook end up in an infinite loop, our application will be stuck and only way to exit will be killing the process.
 

Task Focused Programing with Mylyn and TaskTop

Few days back I came across this presentation at InfoQ site regarding "Eclipse, Mylyn and the TFI". I've seen the Mylyn view in the Eclipse and my idea about that was like some kind of a task management tool that comes with Eclipse IDE. After watching the vide my idea about Mylyn changed a lot and I was really exited about Mylyn and wanted to give it a try.  Mylyn is an open source product, there is a commercial version as well, which is called TaskTop. In task top there is a free version called "TaskTop Starter" which has some more features like Time Tracking.

Installing Mylyn and TaskTop

Mylyn comes by default with many of Eclipse editions. I was using Eclipse JEE Developer edition and it already had Mylyn. So most probably you also having it already. If you don't have Mylyn pug-in already installed in your Eclipse you can get it from Eclipse site. It has all the instruction you need to install it.

Once you install Mylyn you'll get a new view called "Task". If it is not already shown you can get it from Window -> Show View -> Other.

task-view

To install TaskTop starter edition follow the instructions given in TaskTop site. Before installing TaskTop you should do a complete update of your Eclipse. Otherwise you'll get some errors regarding incompatibilities.

Some Interesting Features of Mylyn/TaskTop

Mylyn is all about task focused programing. It allows you to do multitasking with ease. When you create a task all the editors you open are attached to that task. So when you move to another task you can see all the file which you had opened already there. You don't need to waist time on opening and closing editors when switching from one task to another.  Trust me this can improve your productivity many times. I personally experiences this during last tow days

I used mylyn.

task-list

creating-new-task 

You can also integrate bug reports from common bug tracking systems like bugzilla and Jira. Most interesting thing about this is that you can share how you fix the bug with other developers as well. Lett's say you had to open 5 files and do changes in order to fix the bug. When you mark the bug as Fixed you can also attach Mylyn context as well. So when other open it, mylyn will automatically open the files you have opened when you fixed that bug.

Another interesting thing is you can see how you have spent your time on various tasks you have performed during the day. This feature comes with the TaskTop Starter.

time-tracking

There are many more interesting things regarding Mylyn. This is only an enlightment article. If you are interested you should watch that InfoQ video.

 

 

SCTP Client-Server in Java

As I promised in my previous article here is very simple Client Server example for SCTP in Java. If you still have not installed and tested OpenJDK please follow the instructions given in my previous article.

SCTP Server

import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.SocketAddress;
import java.nio.ByteBuffer;

import com.sun.nio.sctp.MessageInfo;
import com.sun.nio.sctp.SctpChannel;
import com.sun.nio.sctp.SctpServerChannel;

/**
* @author sandarenu
* $LastChangedDate$
* $LastChangedBy$
* $LastChangedRevision$
*/
public class SctpServer {

    public static void main(String[] args) throws IOException {
        SocketAddress serverSocketAddress = new InetSocketAddress(1111);
        System.out.println("create and bind for sctp address");
        SctpServerChannel sctpServerChannel =  SctpServerChannel.open().bind(serverSocketAddress);
        System.out.println("address bind process finished successfully");

        SctpChannel sctpChannel;
        while ((sctpChannel = sctpServerChannel.accept()) != null) {
            System.out.println("client connection received");
            System.out.println("sctpChannel.getRemoteAddresses() = " + sctpChannel.getRemoteAddresses());
            System.out.println("sctpChannel.association() = " + sctpChannel.association());
            MessageInfo messageInfo = sctpChannel.receive(ByteBuffer.allocate(64000) , null, null);
            System.out.println(messageInfo);

        }
    }
}

SCTP Client

 

import java.io.IOException;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.SocketAddress;
import java.nio.ByteBuffer;

import com.sun.nio.sctp.MessageInfo;
import com.sun.nio.sctp.SctpChannel;

/**
* @author sandarenu
* $LastChangedDate$
* $LastChangedBy$
* $LastChangedRevision$
*/
public class SctpClient {

    public static void main(String[] args) throws IOException {
        try {
            SocketAddress socketAddress = new InetSocketAddress( 6050);
            System.out.println("open connection for socket [" + socketAddress + "]");
            SctpChannel sctpChannel = SctpChannel.open();//(socketAddress, 1 ,1 );
            sctpChannel.bind(new InetSocketAddress( 6060));
            sctpChannel.connect(socketAddress, 1 ,1);

            System.out.println("sctpChannel.getRemoteAddresses() = " + sctpChannel.getRemoteAddresses());
            System.out.println("sctpChannel.getAllLocalAddresses() = " + sctpChannel.getAllLocalAddresses());
            System.out.println("sctpChannel.isConnectionPending() = " + sctpChannel.isConnectionPending());
            System.out.println("sctpChannel.isOpen() = " + sctpChannel.isOpen());
            System.out.println("sctpChannel.isRegistered() = " + sctpChannel.isRegistered());
            System.out.println("sctpChannel.provider() = " + sctpChannel.provider());
            System.out.println("sctpChannel.association() = " + sctpChannel.association());

            System.out.println("send bytes");
            final ByteBuffer byteBuffer = ByteBuffer.allocate(64000);
            //Simple M3ua ASP_Up message
            byte [] message = new byte []{1,0,3,1,0,0,0,24,0,17,0,8,0,0,0,1,0,4,0,8,84,101,115,116};

            final MessageInfo messageInfo = MessageInfo.createOutGoing(null, 0);
            System.out.println("messageInfo = " + messageInfo);
            System.out.println("messageInfo.streamNumber() = " + messageInfo.streamNumber());

            byteBuffer.put(message);
            byteBuffer.flip();

            try {
                sctpChannel.send(byteBuffer, messageInfo);
            } catch (Exception e) {
                e.printStackTrace();
            }
            System.out.println("close connection");
            sctpChannel.close();

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Technorati Tags: ,,,

 

Communicating over SCTP in Java

SCTP - Stream Control Transmission Protocol is a relatively new (standardize in year 200 by IETF) transport layer protocol to design to operate over IP. It is an alternative for TCP and UDP and combines many good features of those protocols. Main intention of developing SCTP was to cater the growing demand of IP telephony market. 

Since SCTP is relatively new protocol programming language support for this is also not as commonly available as TCP or UDP, especially you are Java developer. But is you are a C/C++ developer then you have no problem, there is a very good guide on UNIX - Network Programing Vol1-3rd Edition (chapter 9, 10 and 23).

If you are Java developer then you'd have to use OpenJDK instead of Sun java JDK, since at present only OpenJDK has the support for SCTP. So first thing you have do is to download and install OpenJDK-7.  Currently it only has versions for Linux and Solaris, again if you are using Windows then you'll have to do some more research. I've tried this only on Linux(Fedora), may be later I'll try with Windows as well. If your Linux kernel do not support SCTP, then you'll have install LKSCTP as well. Here is the small getting started guide at OpenJDK site. You can use the small test program given that guide to test whether your SCTP stack if working properly. I'll post small client-server program in SCTP in a later post.

PS: I found this SCTP library for Windows http://www.sctp.be/sctplib/, but couldn't check whether there is a support from OpenJDK-SCTP side for windows. If you find something please let me know as well.

Technorati Tags: ,,,,
 

OpenCV - Haar Training Resources

There were lot of good feed back for my OpenCV articles (Article1, Article 2). There were lot of inquires regarding Haar Training and how to do it. Since I was bit busy with my work at hSenind I couldn't put any replies for that. Finally I managed to find some time to put together few URL and important tools where you can use to create your own classifier XML.

If you are interested in OpenCV vision library you should join to OpenCV yahoo group. It is very active and has lot of resources as well.  Following are some good links I used when I was learning on how to do Haar Training with OpenCV.

Most important thing to consider when doing Haar training is to have good positive and negative image set. We have created a small utility modifying sample program provide with OpenCV to create positive samples. You can download the source here. Using that you can create positive samples to detect some object using a video. You can move frame by frame in the video using 'Space Bar', mark the areas that contain the object you want to detect using mouse and save it just by pressing 'S'. Utility will create the text file required fro training.

Utility can also be used to create sample images to be used for HMM training as well. If you are interested in HMM following http://www.bernardotti.it/portal/showthread.php?t=17777 has a good article on how to use HMM with OpenCV.

 

Refactoring Messy Code

Few days back I started to refactor some code of a project which I started few years back. Well actually I started working on that project during my second year first semester at the university. It is kind of a pet project which I started with one of my friends. When we started the project, our knowledge about best practices to be used when coding was near zero (I guess it is actually zero).

When I look back the source code of the project, it is like a hell. All possible bad practices are there in our code, very long methods (there is a one method doing some complex calculation which has more than 1350 lines :D ), code duplication, one class doing many things and the list goes on...

Well I can't blame my self for coding this kind of mess, since at that time I was just beginning my journey as a developer and didn't knew any thing about best practices. But I can't keep the code this way since it's readability is 0%; can you imagine reading a method with 1350 lines and understanding. Code has become more messy sine we have included lot of new features and requirement changes here and there. Any how we managed to deal with the code up to now, but now it is overwhelming.

Refractoring is not an easy task, we have to make sure that existing functionalities do not break as well as have to keep up with the new requirements as well.  As Martin Fowler describes in his famous book Refactoring: Improving the Design of Existing Code, there should not be any "Grand redesigns", recatoring should be an incremental process. This is going be be a good experience for me. Let's see how it goes.....

 

Craftsmanship and Ethics

This is nice presentation by Robert C. Martin posted at InfoQ. Here he talk about things that makes us professional. He talk about lot of important topics which help us to do better programming. If you are interested in programming this a going to be really really helpful for you.
In my point of view it is a MUST read for every programmer.
 

diGIT Magazine Launched


First release of the diGIT magazine was launched this month. It is a magazine dedicated for various stuff related to IT. This first release of the magazine is fairly large and rich in content ranging from introductory articles to advance stuff like Agent technology. So everyone from school children to university students can gain a lot from it.
You can view the online version of diGIT or you can download it in pdf format.
I also contributed to this magazine. My article is basically about how to start writing better code. I'm planning to write a series of articles on this topics covering various aspects of better coding practices and methods.