1 2 3 4 5 Previous Next 132 Replies Latest reply on Apr 18, 2011 3:13 PM by brian.stansberry Go to original post
      • 30. Re: Thoughts on filesystem action driven hot deployment
        jason.greene

        Todd Trimmer wrote:

         

        I've been experiencing a problem with atomic moves in AS5 under Windows. About twice a day, AS5's JVM refuses to release a file handle to META-INF/application.xml or an EJB descriptor file. I cannot swap out the old exploded archive when this happens. This is perplexing since it's not like AS5 is hung. All my apps are still working fine.

         

        In light of this, I would much rather have to touch/copy/delete a marker file than rely on atomic moves.

        Thanks for the feedback. The locking semantics on windows are certainly a major problem with the current approaches. So assuming we fixed the access to descriptors, to not hold locks (the correct behavior), delete-undeploy could still fail if there are web requests coming in. Servicing the request requires reading the file, which holds a lock. Granted there is a short lifespan, but all it takes is one happening at the same time to prevent the move.

        Also, while I'm sure the HDScanner is performant, the amount of juggling it has to do seems like a convoluted solution for what shouldn't be such a big problem. But then again, if it were that easy, this thread wouldn't even exist.

        Yeah we want to limit this to be as simple and clean as possible. If we make this more "correct" then we can speed up the scanning time. Part of the reason it ends up with low values is to hide the various race problems. Ultimately though we are limited by what the FS gives us.

         

        Another thing AS7 will have is a CLI, which would be more explicit, intuitive, and immediate e.g. "deploy blah.ear, redeploy blah.ear, undeploy blah.ear". This could even have a maven, or ant plugin on it.

        • 31. Re: Thoughts on filesystem action driven hot deployment
          maxandersen

          Sorry for not seeing this thread earlier, but now that I got pointed to it after trying out JBoss AS 7 Alpha 1 here are my initial good/bad thoughts about this new approach.

           

          First a little about my assumptions:

          JBoss AS 7 is to be usable in todays tools, not the tools of tomorrow. i.e. it should not require magic tools to get a somewhat reliable hotdeployment/development in place.

          The experience one had with AS 4 or even AS 3 or heck, even Tomcat is what I aim for - none of this requires special tooling.

          Special tooling is what is to be used for *improving* things, not as an excuse to say we can't do it without it

           

          Users can't always run their appserver all the time - with AS 7 it might not be a problem because of smaller memory footprint but once users

          start deploying their own stuff to it then that problem returns - hence, most if not all operations should be doable without actually *starting* the appserver.

           

          Local and remote deployment are different things, but both have usecases for hotdeployment functionallity - i.e. me updating a html page shouldn't require a full restart of an app, even if running remotely on an EC2 server for my testing.

           

          Good things in the proposal:

          Using marker file(s) works for me - that is simple to manage and can be done with and without alot of special plugins/tooling, i.e. ant/maven is easy to do it for - harder for IDE's like Eclipse, Netbeans, etc.

           

          I can live with not having automatic rescan of file changes since tools incl. ant/maven can easily create the additional file to trigger deploy.

           

          Bad/Questionable things:

          I don't understand why you have to *remove* the deployed files - if they aren't locked by appserver then please keep them there otherwise tooling have to constantly copy all content to the location (this is not a problem in toy samples, but take a Seam EAR, that is over 20MB of data....that takes time, CPU and disk IO on even the fastests systems - if I touch a html file - tools should not need to copy all again)

           

          For Zipped archives we still have to copy all so that is less of an issue - but why not keep the file there so users can actually see what got deployed and they can adjust content as they need ? (I really find this "delete what got deployed"-approach weird..I don't understand the reasoning)

           

          How are users to know where the actual deployment is happening from ? Suggestion: put a deploy.location=/var/tmp/blahbla/foo.war into the marker file - simply for ease of debugging/transparency.

           

          Why have markers both at toplevel /foo.war.deployed and foo.war/META-INF/jboss-deploy - why not just have them as foo.war.deploy (request deployment of foo.war) and foo.war.deployed (marker for foo.war have been deployed) ? More consistent IMO.

           

          Another issue is though how is all of this going to work for updates that should not trigger a full restart of a module ? i.e. updating html/xhtml pages does not force a restart of a module which causes current sessions to be destroyed ?

           

          With respect to efficient updates/scanning tools (including Ant and maven) could give "hints" to the redeployment by simply listing the files relative to the deployment that actually needs updating - we could put that inside the file marker.

           

          ..and again, its great to have a CLI and i'm all for it but I think we can go very far even without it - not all deployments have to be super-safe nor atomic as long as it goes well in "most" cases

           

          One comment I really find wrong is the following from Jason:

          "Ok so, I will first start off by saying that the file system is a broken API, for many reasons that I will get into, and that we basically have a  bunch of trade-offs here. There is no perfect solution, but I think we  should prefer one that is robust. Ultimately we really should push  people to using the CLI because then we can get truly intuitive  behavior."

           

          or rather, I completely here where this is coming from but the file system is *waay* more accessible and understandable than any CLI we can come up with - it's something we just *have* to support for AS 7 to be easily consumable.

           

          We can always encourage and show by better tooling that the CLI approach is better - but we should still attempt to have a decent hotdeploy operation via file systems - and I don't see why we can't provide that by simply utilizing the marker file concept...


          • 32. Re: Thoughts on filesystem action driven hot deployment
            brian.stansberry

            Thanks for the input. I was off yesterday; hence the slow reply.

             

            Max Andersen wrote:

             

            Local and remote deployment are different things, but both have usecases for hotdeployment functionallity - i.e. me updating a html page shouldn't require a full restart of an app, even if running remotely on an EC2 server for my testing.

             

            I'm interpreting this as a request for a remotable API to allow users to 1) upload exploded deployments and 2) upload individual files to change the contents of those deployments. If that's correct, can you file a JIRA, as that isn't on any roadmap.

             

            I don't understand why you have to *remove* the deployed files - if they aren't locked by appserver then please keep them there otherwise tooling have to constantly copy all content to the location (this is not a problem in toy samples, but take a Seam EAR, that is over 20MB of data....that takes time, CPU and disk IO on even the fastests systems - if I touch a html file - tools should not need to copy all again)

            For exploded content the intent is to leave the original file there. The assumption is the reason people use an exploded deployment is they want to be able to manipulate its contents, so deleting it would be non-sensical.

             

            For Zipped archives we still have to copy all so that is less of an issue - but why not keep the file there so users can actually see what got deployed and they can adjust content as they need ? (I really find this "delete what got deployed"-approach weird..I don't understand the reasoning)

            What do you mean "they can adjust content as they need?" Only thing I can think of is to replace the zip with another version.

             

            I'm OK with leaving the original deployment around. Jaikiran commented way back that removing it forces the user to keep a copy of it around somewhere else.

             

            The logic for removing it is once the scanner has picked up foo.war and deployed it, the foo.war.deployed file is what matters. Deleting that file is what triggers undeploy. Deleting foo.war itself does nothing., and leaving the file there leads people to believe that deleting it means something.

             

            I know that's a bit inconsistent though, since copying in a new version of foo.war does trigger a deployment action.

             

            If we're going to leave the original file around, for zipped files IMO we should do away with the marker files, and use them solely for exploded deployments, where they have a clear purpose. If the zipped files are left around, the only thing the marker files provide is an indication that the content  was actually deployed. There are other ways to see that; and by itself that's not justification for the confusion of having two files around.

             

            How are users to know where the actual deployment is happening from ? Suggestion: put a deploy.location=/var/tmp/blahbla/foo.war into the marker file - simply for ease of debugging/transparency.

            I've been tempted to do that. Haven't because 1) it forms a semi-contract that I'm not sure we should commit ourselves to. 2) It also encourages poking around in places users aren't meant to go poking. 3)  It also forces the deployment content repository to expose internal implementation details to the deployment scanner service, which isn't the end of the world but isn't nice either.

             

            There is INFO level logging of where the deployment content is stored (which shoots my #2 above). :-)

             

            Why have markers both at toplevel /foo.war.deployed and foo.war/META-INF/jboss-deploy - why not just have them as foo.war.deploy (request deployment of foo.war) and foo.war.deployed (marker for foo.war have been deployed) ? More consistent IMO.

            Fine with me.

             

            Another issue is though how is all of this going to work for updates that should not trigger a full restart of a module ? i.e. updating html/xhtml pages does not force a restart of a module which causes current sessions to be destroyed ?

            Good question. We'd been talking in terms of having the exploded content in deployments/ directly mounted, i.e. being directly used as the deployment content. The marker files would serve to trigger deploy/undeploy/redeploy. Once the content was deployed, changes in the content, e.g. a new deployments/foo.war/index.jsp, would be immediately visible.

             

            The downside to that is multiple changes can't be made as an atomic unit, with no requests seeing only some of the changes. But  I don't think that's a critical use case. Devs doing testing will deploy content, make some requests, update some files in the deployment, make some requests, etc. They're not making requests in the middle of copying in changed files.

            • 33. Re: Thoughts on filesystem action driven hot deployment
              maxandersen

              Brian Stansberry wrote:

               

              Thanks for the input. I was off yesterday; hence the slow reply.

               

              Max Andersen wrote:

               

              Local and remote deployment are different things, but both have usecases for hotdeployment functionallity - i.e. me updating a html page shouldn't require a full restart of an app, even if running remotely on an EC2 server for my testing.

               

              I'm interpreting this as a request for a remotable API to allow users to 1) upload exploded deployments and 2) upload individual files to change the contents of those deployments. If that's correct, can you file a JIRA, as that isn't on any roadmap.

               

               

              http://community.jboss.org/thread/154922 list the discussions done at JBoss World and previous about this topic.

               

              If you want me to create a jira for it I will - which component in JBAS should it be ?

              I don't understand why you have to *remove* the deployed files - if they aren't locked by appserver then please keep them there otherwise tooling have to constantly copy all content to the location (this is not a problem in toy samples, but take a Seam EAR, that is over 20MB of data....that takes time, CPU and disk IO on even the fastests systems - if I touch a html file - tools should not need to copy all again)

              For exploded content the intent is to leave the original file there. The assumption is the reason people use an exploded deployment is they want to be able to manipulate its contents, so deleting it would be non-sensical.

              Yup - btw. I assume you copy the content to avoid locks or will the content actually be locked as it was the case for <7 ?

              (i.e. jar's were getting locked on windows 7)

              For Zipped archives we still have to copy all so that is less of an issue - but why not keep the file there so users can actually see what got deployed and they can adjust content as they need ? (I really find this "delete what got deployed"-approach weird..I don't understand the reasoning)

              What do you mean "they can adjust content as they need?" Only thing I can think of is to replace the zip with another version.

               

              In some cases you as a user don't actually have the "original". I.e. when using tools that incrementally keeps the jar uptodate the user just have the "source" files not the actual jar. Meaning if he spots an error somewhere and want to reduce the possible error causes he stops the tooling and go look at the actual deployed thing. Pre AS 7 (or any other server I've worked with) you can simply go and unzip -l the content and from there see what actually got deployed and redeploy again with changes if you need to (without having to involve extra layers of tooling or maintanence).

               

              I'm OK with leaving the original deployment around. Jaikiran commented way back that removing it forces the user to keep a copy of it around somewhere else.

               

               

              Exactly.

               

              The logic for removing it is once the scanner has picked up foo.war and deployed it, the foo.war.deployed file is what matters. Deleting that file is what triggers undeploy. Deleting foo.war itself does nothing., and leaving the file there leads people to believe that deleting it means something.

               

              I know that's a bit inconsistent though, since copying in a new version of foo.war does trigger a deployment action.

               

              If we're going to leave the original file around, for zipped files IMO we should do away with the marker files, and use them solely for exploded deployments, where they have a clear purpose. If the zipped files are left around, the only thing the marker files provide is an indication that the content  was actually deployed. There are other ways to see that; and by itself that's not justification for the confusion of having two files around.

              I agree - if the file stays around the marker file has no real importance.

               

              Well, maybe for consistency and actually knowing the file got deployed without having to scan logs ?

               

              Or is there other/better ways to get this info now ?

              How are users to know where the actual deployment is happening from ? Suggestion: put a deploy.location=/var/tmp/blahbla/foo.war into the marker file - simply for ease of debugging/transparency.

              I've been tempted to do that. Haven't because 1) it forms a semi-contract that I'm not sure we should commit ourselves to. 2) It also encourages poking around in places users aren't meant to go poking. 3)  It also forces the deployment content repository to expose internal implementation details to the deployment scanner service, which isn't the end of the world but isn't nice either.

               

              There is INFO level logging of where the deployment content is stored (which shoots my #2 above). :-)

               

              I keep hearing this "by exposing the actual deployment location there is a semi-contract"-excuse and I just don't buy it.

              The deployment is going to be *somewhere* wether that is on the filesystem or in-memory then telling the user where it is being 'read' from is never a bad information to have.

               

              And as you said - it gets logged anyway (for a reason I assume ?

               

              Why have markers both at toplevel /foo.war.deployed and foo.war/META-INF/jboss-deploy - why not just have them as foo.war.deploy (request deployment of foo.war) and foo.war.deployed (marker for foo.war have been deployed) ? More consistent IMO.

              Fine with me.

               

              At submodule deployments it might make sense to have other markers at a different "Depth" but when its the "same" thing you trigger a redeploy of the marker should be the same.

              Another issue is though how is all of this going to work for updates that should not trigger a full restart of a module ? i.e. updating html/xhtml pages does not force a restart of a module which causes current sessions to be destroyed ?

              Good question. We'd been talking in terms of having the exploded content in deployments/ directly mounted, i.e. being directly used as the deployment content. The marker files would serve to trigger deploy/undeploy/redeploy. Once the content was deployed, changes in the content, e.g. a new deployments/foo.war/index.jsp, would be immediately visible.

               

              Ok - sounds good; but I guess for .jar's you will copy-on-read to avoid classloader filesystem locking or will we still have the same old problem ?

               

               

              The downside to that is multiple changes can't be made as an atomic unit, with no requests seeing only some of the changes. But  I don't think that's a critical use case. Devs doing testing will deploy content, make some requests, update some files in the deployment, make some requests, etc. They're not making requests in the middle of copying in changed files.

              This is actually a critical use case. Today we use jmx calls to tell AS to *not* scan folders while we are copying files over and then continue when we are done. If we didn't do that you would see *alot* more failure deployments.

               

              i.e. users updating a jar or set of jars - that copy operation will actually take some ms's or even seconds for large jars and without disabling the scanner we see alot more partial deployments.

               

              This can happen both on the initial creation of the foo.war or during editing/changes.

               

              I think it would be great have a marker file to prevent deployment - i.e. foo.war.nodeploy would make the scanner skip checking foo.war. Then the tool or user can remove it when he wants it in play again.

              • 34. Re: Thoughts on filesystem action driven hot deployment
                brian.stansberry

                This forum interface drives me nuts. As I write this I'm chatting about what public mailing list we should use for AS 7 dev discussions.

                 

                Re: the remotable API to upload individual files w/in a deployment, let's discuss next week and then I'll open a JIRA.

                 

                Re: Windows jar locking with exploded deployments -- from various discussions I have a vague feeling it may be possible to avoid that without copying.  I'm doubtful. Bottom line is if when this is implemented we have to copy to avoid file locking problems, we'll copy.

                 

                Max Andersen wrote:

                If we're going to leave the original file around, for zipped files IMO we should do away with the marker files, and use them solely for exploded deployments, where they have a clear purpose. If the zipped files are left around, the only thing the marker files provide is an indication that the content  was actually deployed. There are other ways to see that; and by itself that's not justification for the confusion of having two files around.

                I agree - if the file stays around the marker file has no real importance.

                 

                Well, maybe for consistency and actually knowing the file got deployed without having to scan logs ?

                 

                Or is there other/better ways to get this info now ?

                When anything is deployed, an entry for it is written in the standalone/configuration/standalone.xml file.

                 

                How are users to know where the actual deployment is happening from ? Suggestion: put a deploy.location=/var/tmp/blahbla/foo.war into the marker file - simply for ease of debugging/transparency.

                I've been tempted to do that. Haven't because 1) it forms a semi-contract that I'm not sure we should commit ourselves to. 2) It also encourages poking around in places users aren't meant to go poking. 3)  It also forces the deployment content repository to expose internal implementation details to the deployment scanner service, which isn't the end of the world but isn't nice either.

                 

                There is INFO level logging of where the deployment content is stored (which shoots my #2 above). :-)

                 

                I keep hearing this "by exposing the actual deployment location there is a semi-contract"-excuse and I just don't buy it.

                The deployment is going to be *somewhere* wether that is on the filesystem or in-memory then telling the user where it is being 'read' from is never a bad information to have.

                 

                And as you said - it gets logged anyway (for a reason I assume ?

                I have no serious objection to telling people where deployments are stored. This is OSS after all. As we go along there should be a wiki page describing exactly how this works for a given release. I'm just doubtful about persisting it in this file. People come to count on reading the file and you can't lightly stop writing it even if it becomes a problem.

                 

                Why have markers both at toplevel /foo.war.deployed and foo.war/META-INF/jboss-deploy - why not just have them as foo.war.deploy (request deployment of foo.war) and foo.war.deployed (marker for foo.war have been deployed) ? More consistent IMO.

                Fine with me.

                 

                At submodule deployments it might make sense to have other markers at a different "Depth" but when its the "same" thing you trigger a redeploy of the marker should be the same.

                There's no plan to support independent redeployment of submodules.

                The downside to that is multiple changes can't be made as an atomic unit, with no requests seeing only some of the changes. But  I don't think that's a critical use case. Devs doing testing will deploy content, make some requests, update some files in the deployment, make some requests, etc. They're not making requests in the middle of copying in changed files.

                This is actually a critical use case. Today we use jmx calls to tell AS to *not* scan folders while we are copying files over and then continue when we are done. If we didn't do that you would see *alot* more failure deployments.

                 

                i.e. users updating a jar or set of jars - that copy operation will actually take some ms's or even seconds for large jars and without disabling the scanner we see alot more partial deployments.

                 

                This can happen both on the initial creation of the foo.war or during editing/changes.

                 

                I think it would be great have a marker file to prevent deployment - i.e. foo.war.nodeploy would make the scanner skip checking foo.war. Then the tool or user can remove it when he wants it in play again.

                Doesn't the .deploy file mechanism deal with this? With an exploded deployment, the user has to add that file to trigger redeploy. Once the thing is deployed, changing the contents changes what's visible as a resource to the deployment's classloader, but the scanner doesn't do a full redeploy just because it detects some of the content changed.

                 

                I suspect this is leading into the subject of hot replacement of classes, which sounds like a good topic for over a beer next week.

                • 35. Re: Thoughts on filesystem action driven hot deployment
                  dimitris

                  Brian Stansberry wrote:

                   

                  This forum interface drives me nuts. As I write this I'm chatting about what public mailing list we should use for AS 7 dev discussions.

                   

                  Sometimes I'm really buffled why we need to re-invent everything? The main purpose of the mostly-idle jboss-development list (since you are doing most of the discussions over IRC) was to discuss development of and integration into JBoss AS. Plus, all important/interested parties are subscribed to it already.

                  • 36. Re: Thoughts on filesystem action driven hot deployment
                    maxandersen

                    biggest problem with mailing lists is that they are really bad for discussing crosscutting concerns like this deployment challenge

                    • 37. Re: Thoughts on filesystem action driven hot deployment
                      maxandersen

                      Re: the remotable API to upload individual files w/in a deployment, let's discuss next week and then I'll open a JIRA.

                       

                      Re: Windows jar locking with exploded deployments -- from various discussions I have a vague feeling it may be possible to avoid that without copying.  I'm doubtful. Bottom line is if when this is implemented we have to copy to avoid file locking problems, we'll copy.

                       

                       

                      Ok!

                       

                      Max Andersen wrote:

                      If we're going to leave the original file around, for zipped files IMO we should do away with the marker files, and use them solely for exploded deployments, where they have a clear purpose. If the zipped files are left around, the only thing the

                       


                      Or is there other/better ways to get this info now ?

                      When anything is deployed, an entry for it is written in the standalone/configuration/standalone.xml file.

                      ah - so deployment actually changes the original .xml file? Nice.

                       

                      I hope it will retain comments that are in that file ?

                      Why have markers both at toplevel /foo.war.deployed and foo.war/META-INF/jboss-deploy - why not just have them as foo.war.deploy (request deployment of foo.war) and foo.war.deployed (marker for foo.war have been deployed) ? More consistent IMO.

                      Fine with me.


                       


                      At submodule deployments it might make sense to have other markers at a different "Depth" but when its the "same" thing you trigger a redeploy of the marker should be the same.

                      There's no plan to support independent redeployment of submodules.

                      The downside to that is multiple changes can't be made as an atomic unit, with no requests seeing only some of the changes. But  I don't think that's a critical use case. Devs doing testing will deploy content, make some requests, update some files in the deployment, make some requests, etc. They're not making requests in the middle of copying in changed files.

                      This is actually a critical use case. Today we use jmx calls to tell AS to *not* scan folders while we are copying files over and then continue when we are done. If we didn't do that you would see *alot* more failure deployments.


                       


                      i.e. users updating a jar or set of jars - that copy operation will actually take some ms's or even seconds for large jars and without disabling the scanner we see alot more partial deployments.


                       


                      This can happen both on the initial creation of the foo.war or during editing/changes.


                       


                      I think it would be great have a marker file to prevent deployment - i.e. foo.war.nodeploy would make the scanner skip checking foo.war. Then the tool or user can remove it when he wants it in play again.


                      Doesn't the .deploy file mechanism deal with this? With an exploded deployment, the user has to add that file to trigger redeploy. Once the thing is deployed, changing the contents changes what's visible as a resource to the deployment's classloader, but the scanner doesn't do a full redeploy just because it detects some of the content changed.

                       

                      I guess - so if user change a file manually they will have to touch the .deploy file.


                      I suspect this is leading into the subject of hot replacement of classes, which sounds like a good topic for over a beer next week.

                       

                      Luckily I think this problem is actually simpler to handle once we get the basic 'filesystem API' in place.

                      At least from a tooling user/dev perspective - the actual runtime implications is harder

                       

                      • 38. Re: Thoughts on filesystem action driven hot deployment
                        brian.stansberry

                        Max Andersen wrote:

                        When anything is deployed, an entry for it is written in the standalone/configuration/standalone.xml file.

                        ah - so deployment actually changes the original .xml file? Nice.

                         

                        I hope it will retain comments that are in that file ?

                         

                        Sadly, no. I'd love to be able to do that, but we discussed it with PM and agreed it's out of scope for the initial release.

                        • 39. Re: Thoughts on filesystem action driven hot deployment
                          maxandersen

                          initial release meaning this Alpha1 or GA ?

                           

                          Removing comments out of users configuration is an extreme big no-no.

                           

                          Do you at least keep a history of the files somewhere ?

                          • 40. Re: Thoughts on filesystem action driven hot deployment
                            brian.stansberry

                            For GA.

                             

                            We currently keep the previous version around. That's really just a primitive nod in the direction of an appropriate history. We could easily enough keep the original version that was present when the process booted.

                            • 41. Re: Thoughts on filesystem action driven hot deployment
                              dimitris

                              One reminder from the latest meeting: most IDEs will work on the assumption that copying/removing things to "deploy" directories should just work (i.e. no indicator files). So altering that model will break them.

                               

                              In general, the fileystem hot-deployment API seems to be rmore elevant to IDEs/development, so as Max suggested, correctness is less important than convenience and speed of development.

                               

                              I don't know, maybe we could offer an option to have 2 implementations, the old style with its problems, and the new "safe" one.

                              • 42. Re: Thoughts on filesystem action driven hot deployment
                                jaikiran

                                Brian Stansberry wrote:

                                How are users to know where the actual deployment is happening from ? Suggestion: put a deploy.location=/var/tmp/blahbla/foo.war into the marker file - simply for ease of debugging/transparency.

                                I've been tempted to do that. Haven't because 1) it forms a semi-contract that I'm not sure we should commit ourselves to. 2) It also encourages poking around in places users aren't meant to go poking. 3)  It also forces the deployment content repository to expose internal implementation details to the deployment scanner service, which isn't the end of the world but isn't nice either.

                                 

                                There is INFO level logging of where the deployment content is stored (which shoots my #2 above). :-)

                                 

                                I keep hearing this "by exposing the actual deployment location there is a semi-contract"-excuse and I just don't buy it.

                                The deployment is going to be *somewhere* wether that is on the filesystem or in-memory then telling the user where it is being 'read' from is never a bad information to have.

                                 

                                And as you said - it gets logged anyway (for a reason I assume ?

                                I have no serious objection to telling people where deployments are stored. This is OSS after all. As we go along there should be a wiki page describing exactly how this works for a given release. I'm just doubtful about persisting it in this file. People come to count on reading the file and you can't lightly stop writing it even if it becomes a problem.

                                 

                                 

                                I was reading a mail about configuring Java2 security for a web application on AS5 and realized that it might have to play a role here too. Users might have to know the (consistent) location where the deployments will be extracted so that they can setup appropriate permissions.

                                • 43. Re: Thoughts on filesystem action driven hot deployment
                                  maxandersen

                                  Thought I would spend part of my trainride home to outline what I and Brian talked/agreed upon at Devoxx:

                                   

                                  The intent is to have a "safer" file deployment api via the use of marker files.
                                  In case of a deployment named foo.war the following marker files will have meaning.

                                   

                                  foo.war.dodeploy - trigger deploy of archive or exploded directory.
                                  foo.war.donotdeploy - Marker for server to ignore foo.war (used to ensure partial deployments does not happen while copying)
                                  foo.war - in case of archive (not exploded directory) it work as  foo.war.dodeploy if file hasn't changed since last scan/some grace  period.

                                  foo.war.faileddeploy - ACK from server that deploy failed (content should contain info about reason for failure)

                                  foo.war.isdeployed - ACK from server that foo.war.isdeployed.

                                   

                                  There will be flag in server configuration (TBD where exactly) to
                                  enable old style scanning for/reaction on descriptors, but will not be
                                  enabled by default. This will allow existing tooling for pre-AS7 to work sensibly.

                                   

                                  The deployment folder will have a README or similar explaining the above to spread the word on how this folder works.

                                   

                                  ...and that's it

                                   

                                  The following show it in us from a pure command line perspective to show the simplicity (and power) of the "File Deployment API".

                                   

                                  For exploded deployments:

                                   

                                  1. cp -r target/foo.war/ $AS7/deployment
                                  2. touch $AS7/deployment/foo.war.dodeploy

                                   

                                  AS will ignore the directory at step #1 but
                                  step #2 will make it deploy foo.war on the next
                                  scan.

                                   

                                  If deploy is succesfully AS7 will do the following:

                                   

                                  1. rm $AS7/deployment/foo.war.dodeploy
                                  2. touch $AS7/deployment/foo.war.isdeployed

                                   

                                  Meaning that you as user (or tool) can explicitly
                                  control when deployment occurs and you get an ACK
                                  from the server on succesfully deployment.

                                   

                                  In case of error during deployment AS7 will do the following:

                                   

                                  1. echo $DEPLOY_ERROR_INFO > $AS7/deployment/foo.war.faileddeploy (name TBD)
                                  2. rm $AS7/deployment/foo.war.dodeploy

                                   

                                  Where $DEPLOY_ERROR_INFO is what the server knows about the reason for deploy error.

                                   

                                  For archived deployments:

                                   

                                  For archived deployments the behavior/semantics is the same except
                                  AS7 will automatically start deploying an archive once the file has not changed
                                  in some grace period (i.e. 500 ms? )

                                   

                                  The user can trigger redeploy by either doing touch foo.war or touch foo.war.dodeploy.

                                   

                                  "Suspend deployment"

                                   

                                  To handle the case of possible errors occurring while copying deployments (i.e. network connection error
                                  or sudden crash) it is possible to tell the scanner to ignore an archive or exploded directory by doing:

                                   

                                  1. touch $AS7/deployment/foo.war.donotdeploy

                                   

                                  When user want it to be deployed he can simple remove the .donotdeploy and then touch foo.war if its an archive or
                                  touch $AS7/deployment/foo.war.dodeploy in case of directory and archive.

                                   

                                  Info:
                                  We did not use .deploy, .deployed as markers since it could cause confusion in filebrowser that can cut off the ending.
                                  We did not use dodeploy.foo.war since then it would not show up close to the deployment - if you want sorting by "status" sort by extension.
                                  Important that whatever the deployment does theres should be zero delay for reosurces like .html/.xhtml thus shadow copying is probably out of the question; but at the same time need to reduce chance of jar locking on windows filesystems. TBD how and if it is still a problem these days on pre-AS7 deployments.

                                   

                                  Questions:
                                  Will subdirectories be allowed in /deployment for separation ? (don't think so - but good to document that you have to add multiple deployment roots)
                                  If I rm foo.war.isdeployed what does that mean ? Should AS7 create it again ? ( I guess not..they are just transient markers)

                                  • 44. Re: Thoughts on filesystem action driven hot deployment
                                    jason.greene

                                    jaikiran pai wrote:


                                    I have no serious objection to telling people where deployments are stored. This is OSS after all. As we go along there should be a wiki page describing exactly how this works for a given release. I'm just doubtful about persisting it in this file. People come to count on reading the file and you can't lightly stop writing it even if it becomes a problem.

                                     

                                     

                                    I was reading a mail about configuring Java2 security for a web application on AS5 and realized that it might have to play a role here too. Users might have to know the (consistent) location where the deployments will be extracted so that they can setup appropriate permissions.

                                     

                                    A very good point. This should become a separate forum thread, as it affects all deployments (domain deployments) as well. We may need to somehow add this info to the domain.

                                    1 2 3 4 5 Previous Next