Friday, November 13, 2009

Customization and Extension Architecture for Java Web Applications

It is a must that every ERP package has customization architecture. ERP packages are modified according to special requirements of many customers. So we needed customization architecture for our own ERP (composed of many Java EE web applications) too. I analyzed existing ERP packages(i.e. MS Dynamics Customization Architecture) to find out how we can develop such a system for Java EE web applications. There is no real clue from their architecture because of both their programming environment and architecture. I tried to analyze some extension mechanisms within some well-known Java applications like Eclipse. Eclipse plug-in system is great but it doesn’t suit Java web application extension requirements. I read many articles from Internet but there is no satisfactory answer for the problem.

After many years of thinking this issue, I found the answer. Answer was surprising because it was so close to me that I couldn’t see it; Inheritance and Dependency Injection. Here in this post, I’ll present how to use inheritance (thanks to this excellent OOP feature of Java) and DI for application customization and extension requirements.



CUSTOMIZATION TYPES

Throughout this article I’ll use one term “Customization” to mean extension, plug-in or add-on etc. Let’s start with the categorization of customizations. In a typical application, there are 5 types of customizations (Cost increases from top to bottom):

1- By Data (Configuration with Parameters)
The cheapest solution for customization and can be done via system parameter files, or parameter tables.

2- By Workflow and Rule Engine
These systems provide semi-programmatic dynamic logic and can be changed without compilation in the current system on-the-fly. These solutions are cheap compared with programming customizations but it still requires some technical efforts.

3- By Addition
It means adding new applications to the system. To add a new application to existing Enterprise Application is the easiest programmatic customization since it is independent of current applications. In this method, programmer who makes addition may need to use your system libraries. When installing new versions of core applications, binary compatibility should be tested if any library of applications is used.

4- By Extension
Existing core applications functions are modified to meet new requirements without touching the base code. In this method, programmer would need to link base elements in his IDE (otherwise extension may not be compiled). We could support extension for Persistence Objects, Servlets, JSPs, Reports, JavaScript library, Framework and Business Logic. Every component type is extended with different methods, I’ll mention below. In new core application versions, both binary and functional compatibility should be tested.

5- By Modification
In this type, existing core application codes are changed. Programmer would probably need source code of applications (JSPs or JavaScript codes are already open source if not obfuscated). In new application versions, modifications would be lost. Programmer has to apply all modifications in every related new version. This is the most costly customization option.



EXTENSION OF PROGRAMMATIC ELEMENTS

In extension customization type, I said that every programming element can be extended. Here is the complete list of techniques we used for extension:

Persistence Objects
You need 2 things to achieve extension of persistence objects; firstly your persistence object can be extended. Yes, using built-in Java “extends” keyword is not enough so the problem is that does the extended persistence objects work in your Persistence Framework? Second requirement is that you should have a Factory class to return extended object in the run-time. You should build an extension lookup table to understand if this object is extended. If it is extended Factory creates extended object and returns it (DI). In this way, you can change functions of persistence objects by overriding its methods.

//in run-time DBSalesOrderExtension object is created
DBSalesOrder dbSalesOrder = BOFactory.get(DBSalesOrder.class);

public class DBSalesOrderExtension extends DBSalesOrder{
...
}

Servlets
Again you need 2 things to achieve this; firstly there should be no limitation to use extended Servlets in your web framework. Secondly you can forward a request to extended Servlet in “Front Controller” code. Since servlets are created by “Servlet Engines”, you can’t use Factory like persistence objects. Again, you should have a customization definition table to look up extended servlets. Your “Front Controller” checks this lookup table if current Servlet is extended. If so, it forwards, if not it continues processing. Client code or server configurations don’t need any adjustments.

//In Front Controller:
public final void service(HttpServletRequest request, HttpServletResponse response){
...
String extendedSalesOrderServlet = getExtensionName(salesOrderServlet);
if(extendedSalesOrderServlet != null){
doForward(extendedSalesOrderServlet);
return;
}
...
}

public class SalesOrderServlet extends HttpServlet{
...
public void doGet(HttpServletRequest request, HttpServletResponse response){...}
public void doPost(HttpServletRequest request, HttpServletResponse response){...}
}

public class ExtendedSalesOrderServlet extends SalesOrderServlet{
...
public void doGet(HttpServletRequest request, HttpServletResponse response){...}
public void doPost(HttpServletRequest request, HttpServletResponse response){...}
}

JSPs
For JSP extension, you need only one thing; you should have JSP access method(s) to open/redirect JSP pages so that we can inject extended JSP name in these methods. In practice, JSPs can’t be extended with Java’s inheritance mechanism. You can only extend it by copying an existing JSP and modifying its content. Again, you should define its base and extended name within customization lookup table. In JSP access method, you check if target (base) JSP is extended. If extended you redirect to the customized JSP page.

//Original Page
SalesOrder.jsp

//Extended Page
ExtendedSalesOrder.jsp

//In this method, "ExtendedSalesOrder.jsp" is displayed
openPage("SalesOrder.jsp")

Reports
Like JSPs, reports can be extended by copying and modifying it (This is a general approach (I remember same approach within Oracle ADF), if a software element includes GUI elements, inheritance becomes useless for extension). We have company association of reports so we didn’t need a mechanism for extended reports. It is just defined as this company uses this report and company-specific report is executed when requested.

JavaScript Library
We created an empty extension file (js_extension.js) and include it in every JSP page so that JavaScript methods can me modified. In our JS library, everything is method. In JS, there is no direct support for inheritance and overriding. Extender programmer can remove a method in base JavaScript file and add its changed version to this extension file (Overriding or overloading is not possible in JavaScript) or add a wrapper function with different name and use it in his web applications. You can develop a better extension than mentioned here by simulating inheritance in JavaScript.

Framework
Some services can be extended by making new implementations. Here, DI is leveraged again. In our frameworks, we make extension possible in some functions. Implementation is defined within framework configuration file and that implementation is loaded during framework initialization.

public class MyFrameworkSystem{
private IFrameworkService frameworkService;
...
public static void initialize(){
frameworkService = loadImplementation(getImplementationClassNameFromConfFile("FrameworkService"));
...
}
}

public interface IFrameworkService{
public void extensibleMethod();
}

Business Logic
If you need only a fragment of code change or a control is used in many places in base code which are hard to locate by extenders, what do you do? By default, you can extend that object and override it. If that method is a big method, you have to copy all lines to new method and change a piece of it. Maintenance would be a nightmare in that case. A neater solution is to develop an extension injection mechanism. We developed “Extension Points” for just that requirement. If some business logic in base code is going to be modified, we can take it to an extension class to easily override it. In that way, extension can be plugged to any piece of business logic. How can we know which part is customized can’t be answered at the moment. That would be shaped by projects and core customization requirements. Again “Extension Points” should be defined in your customization lookup table. (One interesting case is that you may want to change business logic any time without compilation. Here I mentioned about Rule Engines, you can define a rule and call it in your program. In that way, you can change that rule any time you want; “Dynamic Business Logic”)

void salesOrderBusinessLogic(){
...
SalesOrderBusinessLogic soBL = (SalesOrderBusinessLogic) getExtensionPoint(SalesOrderBusinessLogic.class);
if(soBL.businessLogic())
...
}

public class SalesOrderBusinessLogic implements IExtensionPoint{
public boolean businessLogic(){
//some business logic here
}
}

public class ExtendedSalesOrderBusinessLogic extends SalesOrderBusinessLogic {
public boolean businessLogic(){
//some modified business logic here
}
}



USER CUSTOMIZATION

When talking about application customization, generally user customization is understood. In this sense, you can develop customization options for your applications. You can provide GUI-level user customization facilities. Let’s briefly see what kind of user customizations can be developed:

List Customization
HTML list customization is interestingly difficult (To see that try to change column orders). You may enable your users to customize lists; show/hide columns, column orders, column widths etc.

Input Form Customization
Some user fields may be added to forms by your users. Let’s say your user needs an extra column to track something, if you provide extra column feature (i.e. predefined user columns) they can easily add new fields. Or users may need some default values. For example you can enable them to get last entered data to a form input element. When they re-enter a form and press a shortcut key, their last entered data may be restored to that field again without requiring filling that data.

Report Customization
Your users may need to hide/show some columns etc. Or they may need to add new criteria to report parameter input page. Or they may want to change sort or group columns.

Friday, November 6, 2009

Competing Requirements: Functional versus Non-functional

It is a common situation that can be encountered in many software projects that functional and non-functional requirements intersect. It is generally difficult to make correct decision among these 2 types of requirements. Moreover, some non-functional requirements can also have interrelationships.

Functional requirements are the behavior of software that a user wants to use. Non-functional requirements are invisible attributes that indirectly affects users. These attributes may be sometimes related with developer-side or user-side or system-side. Let’s remember non-functional requirements; Availability, Maintainability, Efficiency, Portability, Flexibility, Reusability, Integrity, Testability, Interoperability, Reliability, Robustness and Usability.

Even when making best selection we have to pay some trade-off cost. In our software requirements review meetings, it is a very usual such a case that an analyst wants a functional feature and programmer rejects for the benefit of non-functional such as performance considerations. What do you do in such conflicts? Performance or a user requirement. Rejecting user requirement is a very easy action but it decrease user satisfaction. We usually start a field analysis, so programmer inspects that feature in his program if a better way would be found. In second meeting, results are evaluated and final decision is taken.

Requirements gathering, elicitation and validation are very precious in software development. Everything begins with requirements. If requirements are not engineered enough, the project is deemed to failure. In this aspect, requirements analysts should be educated and well-informed about both functional and non-functional requirements. In this way, they can filter and design requirement solutions better with non-functional parts in mind. That would only filter some requirements but the final authority for non-functional aspect is programmers and architects.

Sometimes, this competition may lead to personal conflicts. For this kind of situations, we developed a rating system that analysts’ leader has 2-point vote, analyst has 1 point-vote and programmers’ leader has 2-point vote, programmer has 1-point vote and referee (manager) has 3-point vote. We just rate the feature and make the final decision.

To prevent conflicts, another important helper is to declare development priorities. What values do you give prominence as a software team? What are your strategies? Do you plan long-term software? If so, than maintainability would be important then.

You may prepare a “Requirement Documentation“ document that lists best practices of writing requirements for your team. This document may state priorities and make a common agreement for your team.

Links:
Software Requirements

More About Software Requirements: Thorny Issues and Practical Advice

Mastering the Requirements Process

Thursday, October 29, 2009

From API to Solution: My One Year of Technical Blogging Experience

I’ve been blogging for one year. Before starting to blog, I had read many blog experiences to understand the implications of writing a blog. Now, also writing this entry, I found many one year anniversary posts in blogsphere. Everybody lives different experiences about their blogging, me too. At the beginning, I was wondering if I would find enough motivation for writing more than a few posts. Surprisingly, I never missed a week to post during last year. I don’t know whether I’ll keep this sustainability this year.

Naming Blog: “From API to Solution”, what does it mean?
Your first work before blogging would be to find a title for your blog. Although I use different title “Java, ERP, Software Architecture” for my blog, my blog URL is real name of my log and it depicts my professional life; from API to Solution. That means during my profession, I work from micro level (API) to the macro level (Solution). Everything begins with designing an API. Then those APIs composes applications, then applications results products. Products are used via implementation projects. Those projects lead to systems. Systems turn to solutions. I worked in all areas, that name summarize my professional life.

To Blog or Not
If you are a techie like me, what I recommend is if you feel you know something valuable you may blog. Otherwise, don’t write a blog for just to have a blog. When writing something I felt the responsibility of not misleading people when they search something on search engines. If people are losing their time even by only clicking to your buzz blog, you are responsible for the distraction.

Another distraction is to have a joint personal blog with technical one. Sometimes to write about an off-topic becomes very charming, but this would be a waste of time for your readers. You may add some feeling, stories etc. to your posts, but I think this should be limited.

What to Write
Content is the key point that the other people will read your blog. You should share something you know, something you thoroughly analyzed, and something you used. In this sense, my only suggestion is to write only the topics you are sure. Some assumptions, some feelings about technical issues would be easily rejected by your blog commenters. If you can’t defense your ideas, than you feel bad and blogging loses its meaning. Your resistance to commenters would decrease and that would be end of your blog.

How to Write
Before writing a post, my behavioral pattern is like following. Firstly I find a topic and search topic on the net, on the blog posts. If somebody already wrote a post or article about it, I quit that topic and find another topic. If there is no equivalent content on the net, I read what others said about this topic. After reading other articles, I merge with my ideas and experiences and begin to write. I use MS Word for first edit. After writing the post, I read it again for correcting spelling errors and sentence structure errors. Than, I publish it. After publishing, I may change some sentences and add some new paragraphs.

How to Advertise
There are some blog post aggregator sites. You may register there to widen your audience. You may also contribute your posts to some technical portals like java.dzone (Javalobby).

Other Experiences

I don’t like adding ads to the blog: You agree or not, when I open a blog that is full of advertisements, I feel that blog writers’ only aim is to have click stream. Because of that, I have a suspicion about what is written. I feel that ads also make blogs dirty. What I like as a reader is clean and well-organized blogs.

Prefer Suggestion over Criticism: We have some strong beliefs about our technical values. Even tough we state a small criticism against an issue; we may hurt its believers. We’d better use a positive expression way. I got this point after I wrote a criticism post. Needless to say, you must avoid humiliation and curse.

Don’t Repeat Content: I see some blogs that are like news aggregators. If you want to be a web journalist, good luck, but I don’t recommend that. If you want to comment some news, that’s OK but IMHO re-publishing some content again doesn’t add value to readers. I see many blog entries on the net saying same thing, same technical issue, and same observation. Writers should search it before writing something.

Learning by Blog Writing: People say that best way to learn something is to teach it. Same applies to blogging. By writing you also realize what you know again. You check your knowledge by writing, by reading comments, by answering them. First of all, you can learn how to write, how to organize your knowledge to express.

Blogging Requires Time and Motivation: Blogging requires some time dedication. If you don’t have enough motivation, you wouldn’t allocate required time. Many corporations have technical blogs and your employees may tolerate your blog writing during work hours. Motivation may also come from your technical passions.

Blogging Reveals You: When reading technical articles you feel you have a different perspective and that may motivate you to write. By blogging you assert your technical perspective for everyone; your current and future employer, your current and future staff, your customers etc. For this reason, you should write your common denominator ideas.

Blogging in English: Blogging in English provides you to reach a wider audience. Sometimes, people argued me about not writing in my native language. If I’d been writing so, it would be restrained. It is very exciting to share something with people from very distant countries. I study to make my English writing better and beg your pardon for some grammatical and vocabulary mistakes in this blog.