Creating a framework to be used as a base for many other applications

Oh, interesting, received another retrospective just few days after last one. We have developed a base ASP.NET WebFrom framework named ABC and then developed a series of web applications based on it on 2010/2011. One of them is DEF that is currently (late 2016) in production heavily and I guess will be in production at least till 2021 or even 2025 (no retirement plan yet). DEF is dealing with a database more than 10,000,000 records and is used country wide as a national project. DEF is encountering performance problems and is under constant but small changes of the client.

 

Many companies specially those that are not very technical at managerial level love to write code once but use it in many projects more than once. This is why many IT companies have internal frameworks and many people like myself are searching if it is a good idea at all and if yes, what's the best platform to do that. Creating ABC and making DEF and other applications based on it is a good sample for this approach. I'm gonna to review its real weaknesses as it has been in production for few years now.

 

Large Data

ABC has been used as base project of many applications but no one were dealing with large amount of database records as DEF. At the other hand ABC was not designed to deal with that large amount of data. So DEF has performance issues more than other ABC based projects. Performance issues in turn causing other issues relate to save some records in situations that system load is high.

 

Upgrade

As ABC is base framework and many applications based on it are in production and some of them have multiple instances, so upgrading ABC is hard. Suppose I want to upgrade a component in ABC to enhance some features, this upgrade may cause problems in others, at least I can not be sure that this upgrade is safe for all applications. In DEF case we needed to upgrade Nhibernate and did it in painful and very lengthy process.

slow
slow

Internal mechanism

Like upgrade problem, we have difficulties in changing internal mechanisms and designs. For example changing transaction management is somehow necessary for DEF but it must be done through ABC. And as others are using ABC too, it is not an easy way, and sometimes it is impossible to be accomplished. As a result DEF is forced to live with problems that we know why exists and if DEF was standalone, how can be corrected.

 

Do everything through application channel

For a small application that can be run from a shared host, it is not a bad idea if every operation is done via web application itself. But in a large application like DEF there are situations where other tools are needed. For example we have batch operations that take likely 30 or 60 minutes to complete. A good tool is to use windows services to do this type of work. But DEF uses ASP.NET and IIS to do batches that is not good. Many application pool restarts occur during batch or lengthy operations. Also they reduce speed of current logged users and decrease IIS resources and possibly cause secondary problems. Another example is handling a database table with large record count. We handled it hardly in the application while a better way was to introduce a secondary database and define jobs to move old records to it so main database remaining lighter.

inflexibility
inflexibility

Creating packed mega APS.NET controls

If you are familiar with ASP.NET WebForms you know that we have plenty of ASP.NET controls available there, like drop-down-box. In ABC we have had created some mega controls like them to do bigger operations. Think that they were similar to Telerik or Dundas controls, but larger and wider. For example a grid that was able to do paging, sorting and searching. In theory they were very useful and time saving but they were tightly coupled with other internal structure of the ABC and was very inflexible.

 

Conclusion

General purpose frameworks are seen very promising while not used, but in production many cavities are explored. They are good when used in short term and very very similar applications. If you want speed and flexibility think more about “creating each application from scratch” strategy.

Review structure of a web application that I’m working on

There is a work-flow in Scrum called retrospective. It is about reviewing work done in a sprint. I love the idea, I think talking and communication in a software development team is very important. Inspired from scrum retrospective I'd like to have a review on architecture and design of a project that recently I've been involved in. The project is not finished yet but I think it is a good time to review its structures.

 

The project back-end is implemented by ASP.NET MVC Core and Entity Framework Core and is serving both web API and server backed contents (ASP.NET MVC). Develop is done mostly in Ubuntu but also in Windows too.

copyright https://hakanforss.wordpress.com/2012/04/25/agile-lego-toyota-kata-an-alternative-to-retrospectives/

Projects Structure

While project is not very big and complex and have about 20 tables on the database we decided to have 4 projects. One for domain, dtos and contracts named Domain, another for mainly business logic called Core, another for web project itself that contains cshtmls, controllers and wwwroot directory called Web and another for unit tests called Test. I do agree this is a very common project structure for ASP.NET web applications but I saw not benefit over them except than a simple categorizing that also was achievable with directories. I think it was better to have 2 projects, one for web (combining domain, core and web) and another for test.

 

Interfaces

Programming against interfaces are very popular in C#. It has gained more popularity with wide usage of dependency injection in ASP.NET. Also ASP.NET Core have a built in dependency injection that has increased popularity programming against interfaces. We followed this manner in our project. We create an interface per each service class. We have no mockups in our unit tests so I think using so many interfaces are a bit over-engineering because we have no interface in our project that has more than one implementation. Having large number of interfaces just decreased development velocity. Adding each new method needed changes in two place, service itself and interface that it was implementing.

 

Soft Delete

Soft delete means not deleting database records physically, but instead keep them in database and set a filed in each record named IsDeleted to true. So in the application soft deleted records are not showed or processed. We add this feature to the application so we can track data changes and do not allow any data being really loosed. For this purpose we could have used logging mechanism in a way if a record is deleted a log is added that says who deleted what data and when. Implementing soft delete imposed many manual data integrity check to the application. With delete of each record we must check if any dependent item exists or not, if so prevent deletion.

 

Authorization

I personally made authorization over-engineering. I checked roles in both controllers and services in many cases. My emphasis on separation of controllers and services was a bit high. No entry to the app exists other than MVC controllers and API controllers (they are same in ASP.NET Core). So checking roles in controllers in enough.

 

Using dtos to access data layer

Many application designs allow a direct access to database models. For example controllers action get a MyModel instance directly and pass it to dbSet or services to save it. It is dangerous because ORM dirty checking mechanism may save them in the database in mistake. In this project I used a Dto to pass data to or get data from CRUD services. So controllers are not aware of database models. It increased volume of development but I think it saves us from mysterious data updates in the database.

Update:

Second part of this post can be found here.

Software component re-use in ASP.NET and Django and is it really beneficial?

‌I am an ASP.NET developer for years. In many companies and projects that I worked for there is constant need for re-usable components. Some companies were completely serious about it so based their business on it. When I'm doing software projects on my own management (as project manager or as freelancer) I encounter this subject again.

 

Realistic or not realistic, it is very favorable if we could use previously developed components in new projects. It would be very time saving specially in repetitive projects. Code re-use exists in different levels of ASP.NET. You can use Html helpers or user controls to share components between projects, you also can use services like logger or authentication in several projects. There is a type of component reuse in ASP.NET that is used by modules like ELMAH that is based on HTTP modules or middle wares. None of them are a component re-use that I need. What I need is a complete set of component re-use. I need all core and UI elements all together. For example in logger example, I need core logic and all needed UI all together in a manner that I can plug into a new application so other components of the application can communicate an integrate with it. I know there is a solution for ASP.NET that is called Area that approximately do what I need. It do its re-use in view (UI) well. I just copy files into its directory. But I it no designed a really separate component. It is forced to be aware of mother application's internal functionality specially on database design. Maybe it is the reason that ASP.NET MVC area is not very popular.

 

I've read a lot about Django that is re-use friendly by design. I see it is based on apps. Also I see that there is an app sharing web site for it. But never used it in a real project.

 

By thinking more and more on software re-use (in the context of web development) I realize that not every component re-use is suitable for the application. There is trade-off here. If you want to have a re-usable app then you have to develop it as generic as you can. That itself is causing complexity and creating bug and even consumes more time for development. When you start using a component among several projects you must carefully think of every change you made in the application. Each change must be confirmed as backward compatible as others are using your app. So maintenance would be hard. Apparently this is the reason many web development teams do not employ re-usable components a lot.

 

There is at least one situation that this model of software re-use makes sense. When you produce a re-usable app for a limited range of projects and limited period of time and when you are intended to use your app only in a family of project, that would better suites. Here it is good that Django applications are developed in this manner by default, whether you wan to re-use it or not.

Modifying decade-old ASP.NET application

If you are planning to maintain an application for more than a decade then be careful.

I'm in middle of a case of modifying an ASP.NET WebForm application back to 2006 so 10 years old from now. This application is live for now and it is running for no problem. One month ago we migrated its hosting from Windows Server 2003 to Windows Server 2012 R2 successfully. I can imagine if no modification is needed it would be live for another more decade despite vast changing world of web development and .Net.

But our current problem is different than relocating it from a 2003 windows to a 2012 one. We want to modify it so change or add behaviors to it. Some modifications are easy to apply. For example if an HTML/CSS change is wanted it can be done by modifying ASPX/ASCX files easily. But modifications in code behind ASPX.CS and ASCX.CS file are not easy. Because our application is compiled and no CS file exists in hosting IIS. So changes must be applied in source code then published and converted to DLL files then deployed on the hosting IIS. Our application has dependencies to other assemblies like AjaxControlToolkit 2006 and we have not exact version of them currently. So our application simply wont build and wont publish easily.

At first glance I thought the only choice that we have is to upgrade all dependencies to current version that is impossible. From ten years ago til now many of third party libraries have been discontinued or have many breaking changes. Even ASP.NET itself has been changed very much. Fortunately I got a good solution. That was simple and trivial but I didn't noticed it before. We had published version of the application that means we currently have needed assemblies that are referenced from application. No matters if they are very old or we have not access to their source code. They are valid and executable .Net assemblies that can be referenced from any .Net application.

ASP.NET WebForm 2.0 work in Windows Server 2012 R2 again

There is a medium web application in the company that is in use since 2006 to now (2016). This web application is developed with ASP.NET Webform and .Net 2.0 in days of AjaxControlToolkit. For those unfamiliar with that, AjaxControlToolkit and UpdatePanel was an easy way for ASP.NET developers to make use Ajax, the hot technology of the day, in their web applications. Our company web application is hosted in a MS Windows 2003 and now must be hosted to a recent Windows version happened to be MS Windows 2012 R2. This story is about how we manage the application work again in new environment.

The application files copied to a folder in inetpub\wwwroot then an application created for it in IIS. Application was not working with .Net 2.0 App pool because of error “HTTP Error 404.17 – Not Found”. We decided to make it work with a .Net 4.0 App pool. The first error dumped was easy to handle. ASP.Net 4 has some entries of old web.config files into itself so was throwing error duplicate entries. This problem solved by removing those entries from web.config. After it the application get up correctly.

After some usage of the application we explored that some internal tabs are not opening after clicking. Guessing started here. The first guess was that IIS is not serving some file like a .js or a .css file correctly. It was no true because IIS logs and browser profilers did not record any relevant 404 error. Second guess was that something is wrong with AjaxControlToolkit. Our guess was that panels are changed with help of AjaxControlToolkit and UpdatePanel. I checked with Chrome's developer tool. No related console error existed. In another try I changed versions of System.Web.Extensions and AjaxControlToolkit to 3.5.0.0 but didn't helped. Next time added assemblyBinding to web.config but this does not helped too. Event logs of Windows showed no suspicious item too. Several searches in Google neither helped too.

Being hopeless decided to debug client side code of the application in the Chrome developer tool despite of its 4.5K lines of code of legacy ASP.NET WebForm 2.0 application. Started my work by examining JavaScript function that was called by tab headers. javascript:__ShowTAB('ctl00_ContentPlaceHolder1_Panel3',3) was one of those functions. Tracing function calls showed that in an “if comparison” client Ids are not equal. I found it! Client Id generating was changed from .Net 2.0 to .Net 4.0. New one was shorter and didn't had prefix “ctl00_” in beginning of it. A StackOverflow question had same problem. A 2010 post of Scott Gu was showing the complete cause of the problem. ASP.NET 4 WebForm has been moved along a way to generate a cleaner client id. This behavior could be controlled via ClientIDMode. Unfortunately default value of this property was not AutoID while ASP.NET 2.0 behavior was as same as AutoID. I added ClientIDMode = AutoID to the directive of the master page and the problem solved peacefully.