Search This Blog

Monday, December 31, 2012

THE Framework - Iterative Elements FAIL

   Iterative elements are not going well.  Here are notes on this:  

31 Dec., 2012
 ------------------  
   Worked on the iterative problem.  I tried creating a custom foreach Directive for Velocity.  First, the Parser will not accept an underscore in the directive name.  I had wanted to use the_foreach, but that's no good.  Second, the Parser will only allow the argument syntax ( $item in $list ) in the foreach Directive. It's a hard coded literal in the Parser.

   So, I tried creating a subclass and/or a wrapper for the foreach Directive.  The name can't be anything but foreach.  The next effort was a subclass that copied the foreach Directive code entirely and modifying from there.  This required creating a ForeachScope wrapper class as well to allow access to the private members of the scope class.  The class uses the foreach name and is found before the Foreach class by the Parser.  This is fragile.  It does allow additional arguments to be passed in.  It accepts an optional boolean argument to indicate whether to perform the nesting constructions.  All of this could break with changes to Velocity.

   The idea was to have a variable called currentLabel and a stack of currentLabels in the Velocimacro library. On the call of a nested foreach, the currentLabel would be pushed on the scope stack.  The currentLabel would have the value of the list argument appended to it.  As the loop ended, the currentLabel would be replaced by popping the top value off the stack.  This allows nested scoping.

   Inside the body of the foreach loop, tags that needed to use the currentLabel would call a VelociMacro called nestedBindTo.  This would prepend the currentLabel to the property name and place it in the bindTo attribute.

   Here's the problem I'm trying to solve.  The controller's model has a List of something, say of Object Group. Each Group wants to be a Panel or Accordion or table row.  Each Group has a List of Items.  These want to be in a table or ordered/unorder list.  They want to be able to subscribTo or reRenderOn events, so they need to be bound back to the controller's Model by the JavaScript engine.  The end rendered result would look like this:  

<ul id="groups" subscribeTo="save" bindTo="groups">
   <ul subscribeTo="save" bindTo="groups.0">
      <li><input type="text" subscribeTo="save" bindTo="groups.0.item.0" value="Exciting Item One"></li>
      <li><input type="text" subscribeTo="save" bindTo="groups.0.item.1" value="Boring Item Two"></li>
   </ul>
   <ul subscribeTo="save" bindTo="groups.1">
      <li><input type="text" subscribeTo="save" bindTo="groups.1.item.0" value="Group 2 Item One"></li>
      <li><input type="text" subscribeTo="save" bindTo="groups.1.item.1" value="Group 2 Item Two"></li>
   </ul>
</ul>  

   I could finish my class hack on the foreach Directive, but this is a terrible hack and very fragile.  Another option would be to dump the responsibility for nesting on Devlopers.  This puts a load of tedious mapping on the Developer and is error prone.  I would still provide the currentLabel, scope stack, and nestedBindTo VelociMacro as well as push and popAndSetCurrentLabel methods for the stack.  Coding would have to look like this:  

#set( $currentLabel = 'groups' )
<ul id="groups" subscribeTo="save" data="#bindTo( 'groups' )">
#foreach( $group in $page.groups )
#push( $currentLabel )
#set( $currentLabel = $currentLabel'.'$foreach.index )
   <ul subscribeTo="save" data="#nestedBindTo( $currentLabel )">
   #foreach( $item in $group )
      #push( $currentLabel )
      #set( $currentLabel = $currentLabel'.'$foreach.index )
      <li><input type="text" subscribeTo="save" value="#nestedBindTo( $currentLabel )"></li>
      #popAndSetCurrentLabel()
   #end
   #popAndSetCurrentLabel()
#end  

   This points out a number of flaws in the design.  It does not handle non-form elements having a binding where the value gets displayed.  As an example, what if the list items above want to have their values displayed as simple text?

<li data="#nestedBindTo( $currentLabel )"> --- How to get and show the value here? How will it be reRendered? --</li>

This wants a completely different solution.

Sunday, December 30, 2012

Something Different - THE Framework

   So I find myself with a little time off here at the end of the year.  I'd been wanting to try out a few of the many Java web frameworks out there.  I'm always looking for "the one".  You know, the one that does everything you want effortlessly with no configuration and no glitches.  The one that will handle all those outside cases that other frameworks make you really hack the heck out of them to get it work.  Yeah, that impossible framework.  I checked out the latest Tapestry, tried Wicket, looked at Grails.  They all frustrated me fairly quickly.  I was looking for immediate solutions to problems I've faced with JSF, facelets, RichFaces, IceFaces, ADF, and Struts.  With lots of free time, I started thinking about what that ideal framework would look like.

   I started with the usual principles: simplicity and flexibility.  I assume that the developer using the framework is like me: intelligent and intelligently lazy.  This developer doesn't want to have his hand held or be tied down into one style of solution.  This developer also doesn't want to have to think to much about the framework or its configuration.  He wants it to allow rapid prototyping that can be quickly adopted as working code.

   Oh yeah, and AJAX, lots of AJAX.  I mean, really tight AJAX requests and responses.  Each submit should send exactly and ONLY what it needs to send.  The responses should have just the stuff needed, too.  Any tag should be able to register itself as a listener for either or both the send and response to any event.

   The events should be able to register themselves, including their name, and what phases they want to use.  The phases would be: gather submit data, send a request, and reRender.  The event could even skip the AJAX send if it wants to and just be a gather data and reRender.  This would be useful especially in development.

   The listeners would register themselves just by adding attributes to their tags.  To register to be sent to the server, it would add a subscribeTo attribute with the names of the events it wants to be a part of ( of which it wants to be a part, for the grammar nazis ).  To register for the reRendering, it would add a reRenderOn attribute with the names of the events of which it wants to be a part.

   Now, here's the cool part.  In the value attribute for form fields, or the data attribute of ANY tag, it would simply put something like 'value="page.person.address.lineOne"'.  If the page defined a model Object for person.address.lineOne with a value like '227 Main St.', the engine would alter that on the server side to be 'value="227 Main St." bindTo="person.address.lineOne"'.  This would help the client side JavaScript engine to take the value or data attribute's value and map it to a JavaScript Object graph of person.address.lineOne.  The AJAX request would send this as a JSON string to the server.  The server would automagically transform this to the page's Object graph.

   Once it had processed the request, the server would send an (optional) JSON response.  The client side JavaScript engine would receive this and map the response back to the tags that had registered for reRendering.

   A templating engine would allow page composition and fragment reuse.  Requests for a new page (navigation) would use a POST/Redirect/GET(POST) paradigm.

   Above all, the framework would allow maximum flexibility.  Provide a means to register and de-register listeners on the fly.  It would intercept true submits and turn them into AJAX requests with all values from all bound tags with subscribeTos, but this would be a convenience.  Developers would have to include a few key things in their controllers to make things work.  A PageMaker would handle the necessary alterations to the templates to add the bindTo attribute.

   One sticky part was the amount of raw configuration that would be required.  A Maven archetype could set up most of the necessary boilerplate, saving the developer a lot of time.  The rest is down to documentation.

   Could this really work?  I decided to play around.  The results aren't fantastic yet, and they have already led to a number of less than ideal solutions.  On the whole, it looks pretty cool, and it has been a lot of fun to do.  Here's what I came up with.

   My name is Thomas Edward Hunt.  My monogram, like the kind you get from your aunt on a set of hankies or an ugly sweater, is THE.  I decided to create THE framework.  It's really more of a stack with some glue code.  For the client side, I went with jQuery and jQuery-UI.  It has all the selector and AJAX functionality I was looking for, along with all the cool UI stuff.  For the templating engine, Velocity seems to have everything required.  It allows page composition.  It can do the necessary attribute instrumentation ( I think ).  I still have a nagging problem with iterative elements.  See my notes below for more.  For the servlet piece, Jersey has proven very flexible.  With JaxB, it can take JSON as an argument and render JSON from an Object graph with little or no effort.  Just a little effort gets you the POST/Redirect/Get paradigm.  Spring adds the necessary glue for almost everything.  Spring Acegi will do the security.  Add in the ORM of your choice.  I haven't even begun to look at validation or submission value scrubbing/security.  One thing at a time.

   Here are my working notes to date.  These are REALLY rough, but I like to capture my evolving thoughts.

*********************************************************************************
NOTES:
*********************************************************************************


18, Dec., 2012
---------------

Premise:
Build a web application in Jersey and jQuery.  The majority of requests will be Ajax calls that will return a JSON object.  The HTML will be mostly plain markup.  A special attribute on tags will mark what events it will subscribe as a listener for refresh.  Whole swaths of HTML may be rewritten in this way while still allowing prototype placeholder text to be put in the HTML.  Gotta figure out how to do this.

Jersey will allow composition of HTML by recursively scanning the page templates for composed sub-pages.

On Ajax response, jQuery should allow to find all elements whose subscribe attribute is set to the event at hand.

This wants to be converted to a JSON object. Ajax wants to send a POST of the JSON to Jersey.  Jersey will convert that to an expected POJO and process.

Server side tasks:
1. Receive request for page.  Get master template.  Recurse sub templates and compose page.  Parse EL and fill with markup.  Blech! (Antlr?)
2. Receive JSON with a request as a POST.  Convert JSON and process request.  Return JSON or FORWARD to new Resource.

Client Side tasks:
1. build Subscription lists.  Probably 2 lists: one for "I'm a part of submits for" and one for "I'm a part of reRenders".
2. Build Ajax framework to get all sendSubscribed elements from an Ajax submit and build JSON object.  Submit Ajax request as a POST.
3. Ajax framework to receive all responses.  Parse JSON response.  Need a way to know where each element of JSON goes  in the DOM without encumbering the server with knowing about the page's needs.
4. Intercept full page submits and still perform the mappings needed.
5. Need a way for each page to intercept all requests and responses for pre- and post- processing.  Ya just need it.


#####################################################################
FORWARDS IN JERSEY
#####################################################################

Step 1. Get access to HttpServletResponse. To do it declare in your service something like:

@Context
HttpServletResponse _currentResponse;
Step 2. make redirect

...
_currentResponse.sendRedirect(redirect2Url);
EDIT

Well, to call forward method you need get to ServletContext. It is can be resolved the same way as response:

@javax.ws.rs.core.Context
ServletContext _context;
Now _context.forward is available



20 Dec., 2012
------------------

Okay, Here is the stack idea: javascript: jQuery and Bootstrap.js; View: Velocity;  Servlet Engine: Jersey ( need  to marry with VelocityViewServlet in some way.  Custom: event registration for send and reRender, using jQuery.  This will need the rewrite in velocity to add attributes of subscribe and reRender as well a tag
that will indicate to what part of the model a tag belongs ( eg, person.address.firstLine ). In other words,
it will need to extend the basic Velocity engine to include adding in the modelRef attribute to point to the
model path ( eg, person.address.firstLine ).

Engine: Jersey: will take in a JSON and put out a JSON ( figure out how to accept and auto-translate JSON
to Java ).  Figure out page transitions with Post-Redirect-Get.  Figure out how to return Velocity templates.

Wire to Spring, Acegi, Hibernate, etc. and make an archetype.  Figure out a Submit intercept to follow regular AJAX course.

Velocity looks like it is extensible.  It definitely supports includes, so that' good.

Need to figure out the VelocityViewServlet and Jersey integration.  It wants this to be seamless yet extensible. Not sure how to do this.  Also, needs a security measure to ensure against cross-site scripting, SQL injection, etc.



26 Dec., 2012
------------------

First, Jersey will take a JSON as input and parse it to the Models it knows.

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
GENERAL APPROACH GUIDELINE
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This wants ANY tag to be a part of the conversation with the server.  To do this, the general approach will be that the value bound will be the the value attribute if the element involved is an form element such as an input, a select, or a textarea.  If the element is not one of these, it should look to a data attribute for the
value.

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

   Now, the tags want two attributes: subscribeTo and reRenderOn.  These will have a comma separated list of events in them.

   On render of the page, the event handler will somehow magically insert an additional attribute for any element that has one or both of these tags.  This extra attribute is the bindTo.  It will have the path to the model field such as person.address.lineOne.

   This will need to be modified to accomodate iterative things like lists, table rows, dropdown menus, etc.  They may look like category.0, category.1, etc.

   In the jQuery onload process, there wants to be a method to register all listeners.  The idea is to get a list of all elements that have a bindTo attribute. Simple to do with a selector: $('[bindTo]').

   Each Ajax event wants to have an event name.  This could either be an attribute on the calling element ( button, link, etc ) or a string passed in as a parameter.  The latter seems to offer more flexibility.  Each event
will have several phases.  There should be a means to indicate which phases the event requires.  The phases will be
1. Gather data from registered elements with a subscribeTo that contains the event and form an Object from it.
2. Transmit an Ajax request ( possibly with a JSON Object from step 1 ) to the server.
3. Handle reRendering, based on registered elements with the event in their reRenderOn attribute.

Really, it would be cool and very flexible if the event could specify any combination of these three phases.  Additionally, the event should be able to pass in an arbitrary Object to use in place of the model.  This can be added later.

   The call first wants to get a list of all registered elements that have the event name in their subscribeTo attribute.  It wants to retrieve the value of each element.  The value will be the value attribute if the element is a form element.  Otherwise, the value will be in the data attribute.  The function will use the element's bindTo in order to add the value to the JSON object that will be passed to the server.

   Once the JSON Object has been formed and stringified, a jQuery.post() is made.  The callback will handle the response and reRender.

   The callback wants to get a list of all registered elements that have the event name in their reRenderOn attribute.  It wants to retrieve the value from the JSON Object in the Ajax.data, being mindful of iterative indexes in the bindTo value.  It wants to set the new value in the element.  It needs to figure out how to make sure that the visible page changes when this value is changed.
Notably, dropdown lists have been problematic.


Runtime gives EventCartridge gives Iterator over ReferenceInsertionEventHandlers gives events I want to mess with ( I think ).



27 Dec., 2012
------------------

Decided to not use Bootstrap.js.  It is just not usable at this point.  Documentation is lacking and it doesn't
work with IE 7-8.  Reality is, a good number of users still use these sub-standard browsers.

Decided to go with jQuery UI.

Odd issue with Accordion under a tab nav.  Height of accoridions is zero.  Google, please...

StackOverflow tells us: you must initialize the accordion before the tabs!!!  Gack!  How crude, but it works...

Best practice with jQuery MAY BE ( don't know ) to load all inner widgets before the outer ones.

Got the initial project set up with Spring, Jersey, Velocity, MyBatis.  Very primitive but working.  Next, need
to add velocity template composition, get sources, and figure out event handling...



29 Dec., 2012
-----------------
Okay, Got jQuery, jQuery-UI, Jersey, Velocity, and Spring set up.  Need to bind Velocity into Spring eventually.

Wrote the PageMaker class to handle rendering page templates for all Pages.  They just need to extend PageMaker.
May want to change this, so it doesn't consume a Page's only super class...  Maybe an injected utility?
It really doesn't want to be a static class.

Dropped the EventHandler solution.  The event didn't have enough context.  Velocity's parser is an actual language parser from JavaCC.  It lexes a stream into tokens, parses the token stream into an AST, and then walks the AST to generate the page from the template and values.

Instead, THE provides a Velocimacro in the library.  This macro is available to all templates.  Developers will need to use the macro on anything they want mapped to the server's model.  For form fields, this will be the value attribute.  Any other tag can be bound with the data attribute.  For a form field that wants to bind to, for example, HomePage's person.address.lineOne, the developer would put this in the vm file:

<input id="lineOne" type="text" value="#bindTo( 'page.person.address.lineOne' )" />
 
    Note, that the value passed in is a string literal.  It has NO Velocity
    variable indicator.  The macro will turn this into a variable reference and also
    use the value for its bindTo value.  Likewise, you only need to ever specify
    'page' for the page and not the page's actual name.  This is handled by the
    PageMaker class.  The macro actually strips out the "page." part of the string.
That's why PageMaker provides the universal "page" root.

The rendered result would be:
<input id="lineOne" type="text" value="227 Foo St." bindto="person.address.lineOne"/>



30 Dec., 2012
------------------

Moved PageMaker to be an injected class ala Spring.  Developers will have to inject this into their pages or just new it up as needed.  It has no dependencies as yet, so either way will work.  The framework archetype injects it for now in case it adds dependencies later.

Next Steps:

1. Work on page navigation and loading.  Looks like Post/Redirect/Get might be the right strategy here.
    The client side probably wants to intercept full submits, grab everything registered for any subscribeTo event and push it's current value into the javascript Object that will then be turned into JSON.  The default submit will be suppressed, and an AJAX POST will be made instead.  The server will receive and process  as needed, then send a redirect.  This **SHOULD** result in a second GET request for the new page.

The redirect will need to have any needed attributes loaded into the query string.

For later consideration: Should this be a Post/Redirect/POST?  Developers could put sensitive stuff in the request...

2. Client Side:  Lots of good javascript to write here.  Baby steps here.
A. get the registration of event listeners done.  This is a list of any elements with a bindTo attribute set with either a subscribeTo or reRenderOn attribute.

NOTE: THE VELOCIMACRO SOLUTION PRESENTS A SERIOUS PROBLEM FOR ITERATIVE ELEMENTS!  Sorry to shout, but this needs attention.  A separate macro will hopefully solve for this.  The whole solution is looking a little shaky.

 The development experience is getting a little too involved in the engine.  This really wants to be abstracted  from the developer.

B. Figure out event registration, including phases required and callbacks. Populate the Object for transmission.

C. Handle the reRender callback.

D. Add methods for registering and un-registering listeners on the fly.


*********************************************************************************
*********************************************************************************

A good portion of the JavaScript in the last TODOs has already been worked out in a proof of concept.  I'm now working on getting it to a more robust and true solution.  The biggest sticking point is the iterative components.  I'm not quite sure whether to figure that out first or just move on the JavaScript solution.

I'd appreciate any feedback.  Again, this was just a thought experiment that turned into a bit of code as an exercise.  As long as it keeps feeling worth it, I'll keep pursuing it.  If someone has already solved a part of this well, let me know.  I'd be glad to save a few hours on something that's already solved.  I'm already leveraging a huge stack of code to solve well defined problems.

Thanks for reading!

Tom