Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

uwolfer
I sufferred a problem which sounds almost like this problem. After
updating to 3.0.1 (from 3.0.0-rc3-20120401), the application did not
respond for a long time after a restart (sometimes several minutes).

I tried to find out which commit introduced and was able to identify
revision 3908 as breaking change for me. After looking into all changes
done in this change, I've found out that the following change
introduced my problems:
-  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
+  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];

There is a way to overwrite this setting in your application code:
dwr.engine.setRetryIntervals([ 1, 1, 10 ]);

It sounds really strange that just this change causes such issues.
While investigating a bit more, I have seen that the server "bombs" the
server with a lot of "__System.checkHeartbeat.dwr" and
"ReverseAjax.dwr" requests in a short time while the server is
starting. All these requests keep the state pending and take several
seconds to more than one minute to finish. While they are pending, dwr
issues even more requests of this type.

Any ideas? I think you should not issue any more polling requests while
requests are pending.

One minor note: the setting change should also result in an update of
the comment: "Retry every 10 seconds when offline." (you have changed
it to 3s).

Bye
urs
Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

david@butterdev.com
The reporter of this issue must have messaged me privately and indicated
the problem was with Jetty and their servlet config.  I will look up his
message and post it to the list.

I also find it strange that this is the root cause of your problem.   We
have had thousands of downloads of 3.0 and no reports of something
similar. The calls in question are heartbeat calls and should be
executing extremely quickly, even if several of them queue there should
not be a problem.  If you can minimize your code and send us a war that
reproduces the problem that would be helpful.

You are correct about the comments in engine.js.  I will update those
and also clarify the retry docs a bit
(http://directwebremoting.org/dwr/documentation/reverse-ajax/retry.html).


On 2015-12-21 00:38, [hidden email] wrote:

> I sufferred a problem which sounds almost like this problem. After
> updating to 3.0.1 (from 3.0.0-rc3-20120401), the application did not
> respond for a long time after a restart (sometimes several minutes).
>
> I tried to find out which commit introduced and was able to identify
> revision 3908 as breaking change for me. After looking into all changes
> done in this change, I've found out that the following change
> introduced my problems:
> -  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
> +  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];
>
> There is a way to overwrite this setting in your application code:
> dwr.engine.setRetryIntervals([ 1, 1, 10 ]);
>
> It sounds really strange that just this change causes such issues.
> While investigating a bit more, I have seen that the server "bombs" the
> server with a lot of "__System.checkHeartbeat.dwr" and
> "ReverseAjax.dwr" requests in a short time while the server is
> starting. All these requests keep the state pending and take several
> seconds to more than one minute to finish. While they are pending, dwr
> issues even more requests of this type.
>
> Any ideas? I think you should not issue any more polling requests while
> requests are pending.
>
> One minor note: the setting change should also result in an update of
> the comment: "Retry every 10 seconds when offline." (you have changed
> it to 3s).
>
> Bye
> urs
Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

zooppoop
Sorry I have not responded I have not had time to work on the debuging
code.  I still have the same issue as before but am trying to do as
suggested putting in stack traces where you had requested.  Or getting
some boiled down code that can replicate the issue.    I will also try
the change that Urs suggested see if it fixes the problem for me.   Urs
are you saying you set this on page load in JS I assume?

Thanks.


21.12.2015 08:21 に [hidden email] さんは書きました:

> The reporter of this issue must have messaged me privately and
> indicated the problem was with Jetty and their servlet config.  I will
> look up his message and post it to the list.
>
> I also find it strange that this is the root cause of your problem.
> We have had thousands of downloads of 3.0 and no reports of something
> similar. The calls in question are heartbeat calls and should be
> executing extremely quickly, even if several of them queue there
> should not be a problem.  If you can minimize your code and send us a
> war that reproduces the problem that would be helpful.
>
> You are correct about the comments in engine.js.  I will update those
> and also clarify the retry docs a bit
> (http://directwebremoting.org/dwr/documentation/reverse-ajax/retry.html).
>
>
> On 2015-12-21 00:38, [hidden email] wrote:
>> I sufferred a problem which sounds almost like this problem. After
>> updating to 3.0.1 (from 3.0.0-rc3-20120401), the application did not
>> respond for a long time after a restart (sometimes several minutes).
>>
>> I tried to find out which commit introduced and was able to identify
>> revision 3908 as breaking change for me. After looking into all
>> changes
>> done in this change, I've found out that the following change
>> introduced my problems:
>> -  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
>> +  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];
>>
>> There is a way to overwrite this setting in your application code:
>> dwr.engine.setRetryIntervals([ 1, 1, 10 ]);
>>
>> It sounds really strange that just this change causes such issues.
>> While investigating a bit more, I have seen that the server "bombs"
>> the
>> server with a lot of "__System.checkHeartbeat.dwr" and
>> "ReverseAjax.dwr" requests in a short time while the server is
>> starting. All these requests keep the state pending and take several
>> seconds to more than one minute to finish. While they are pending, dwr
>> issues even more requests of this type.
>>
>> Any ideas? I think you should not issue any more polling requests
>> while
>> requests are pending.
>>
>> One minor note: the setting change should also result in an update of
>> the comment: "Retry every 10 seconds when offline." (you have changed
>> it to 3s).
>>
>> Bye
>> urs
Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

david@butterdev.com
Sorry, I got confused with your other thread - "DWR 3.0 Build 646 and
Springs 4.2.3"

Mike didn't suggest you put in stacktraces (whatever that means). He
suggested you download the DWR source and put some breakpoints in the
code to see what the call stack is when the following warning is logged
(per your original message) - "org.directwebremoting.extend.Factory  -
DWR has not been initialized properly"

Urs did not mention getting any errors in the logs so it is quite
possible these issues are unrelated.

On 12/21/2015 05:32 PM, junkmail wrote:

> Sorry I have not responded I have not had time to work on the debuging
> code.  I still have the same issue as before but am trying to do as
> suggested putting in stack traces where you had requested.  Or getting
> some boiled down code that can replicate the issue.    I will also try
> the change that Urs suggested see if it fixes the problem for me.
> Urs are you saying you set this on page load in JS I assume?
>
> Thanks.
>
>
> 21.12.2015 08:21 に [hidden email] さんは書きました:
>> The reporter of this issue must have messaged me privately and
>> indicated the problem was with Jetty and their servlet config. I will
>> look up his message and post it to the list.
>>
>> I also find it strange that this is the root cause of your problem.
>> We have had thousands of downloads of 3.0 and no reports of something
>> similar. The calls in question are heartbeat calls and should be
>> executing extremely quickly, even if several of them queue there
>> should not be a problem.  If you can minimize your code and send us a
>> war that reproduces the problem that would be helpful.
>>
>> You are correct about the comments in engine.js.  I will update those
>> and also clarify the retry docs a bit
>> (http://directwebremoting.org/dwr/documentation/reverse-ajax/retry.html).
>>
>>
>>
>> On 2015-12-21 00:38, [hidden email] wrote:
>>> I sufferred a problem which sounds almost like this problem. After
>>> updating to 3.0.1 (from 3.0.0-rc3-20120401), the application did not
>>> respond for a long time after a restart (sometimes several minutes).
>>>
>>> I tried to find out which commit introduced and was able to identify
>>> revision 3908 as breaking change for me. After looking into all changes
>>> done in this change, I've found out that the following change
>>> introduced my problems:
>>> -  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
>>> +  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];
>>>
>>> There is a way to overwrite this setting in your application code:
>>> dwr.engine.setRetryIntervals([ 1, 1, 10 ]);
>>>
>>> It sounds really strange that just this change causes such issues.
>>> While investigating a bit more, I have seen that the server "bombs" the
>>> server with a lot of "__System.checkHeartbeat.dwr" and
>>> "ReverseAjax.dwr" requests in a short time while the server is
>>> starting. All these requests keep the state pending and take several
>>> seconds to more than one minute to finish. While they are pending, dwr
>>> issues even more requests of this type.
>>>
>>> Any ideas? I think you should not issue any more polling requests while
>>> requests are pending.
>>>
>>> One minor note: the setting change should also result in an update of
>>> the comment: "Retry every 10 seconds when offline." (you have changed
>>> it to 3s).
>>>
>>> Bye
>>> urs
>

Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

zooppoop
Yea that is what I was trying. I did think he asked for a stacktrace.  
Either way it is about the same what I was doing just adding a
printStackTrace there.  Odd thing is with my test Code it does not even
seem to be hitting that function.  Since I got to that point I have not
had time to dig into it more.  I'll definitely get back to it and
respond with what I find so if anyone else runs into it they have some
info. ;)

Thanks.


21.12.2015 18:12 に David Marginian さんは書きました:

> Sorry, I got confused with your other thread - "DWR 3.0 Build 646 and
> Springs 4.2.3"
>
> Mike didn't suggest you put in stacktraces (whatever that means). He
> suggested you download the DWR source and put some breakpoints in the
> code to see what the call stack is when the following warning is
> logged (per your original message) -
> "org.directwebremoting.extend.Factory  - DWR has not been initialized
> properly"
>
> Urs did not mention getting any errors in the logs so it is quite
> possible these issues are unrelated.
>
> On 12/21/2015 05:32 PM, junkmail wrote:
>> Sorry I have not responded I have not had time to work on the debuging
>> code.  I still have the same issue as before but am trying to do as
>> suggested putting in stack traces where you had requested.  Or getting
>> some boiled down code that can replicate the issue.    I will also try
>> the change that Urs suggested see if it fixes the problem for me.  
>> Urs are you saying you set this on page load in JS I assume?
>>
>> Thanks.
>>
>>
>> 21.12.2015 08:21 に [hidden email] さんは書きました:
>>> The reporter of this issue must have messaged me privately and
>>> indicated the problem was with Jetty and their servlet config. I will
>>> look up his message and post it to the list.
>>>
>>> I also find it strange that this is the root cause of your problem.
>>> We have had thousands of downloads of 3.0 and no reports of something
>>> similar. The calls in question are heartbeat calls and should be
>>> executing extremely quickly, even if several of them queue there
>>> should not be a problem.  If you can minimize your code and send us a
>>> war that reproduces the problem that would be helpful.
>>>
>>> You are correct about the comments in engine.js.  I will update those
>>> and also clarify the retry docs a bit
>>> (http://directwebremoting.org/dwr/documentation/reverse-ajax/retry.html). On 2015-12-21 00:38, [hidden email] wrote:
>>>> I sufferred a problem which sounds almost like this problem. After
>>>> updating to 3.0.1 (from 3.0.0-rc3-20120401), the application did not
>>>> respond for a long time after a restart (sometimes several minutes).
>>>>
>>>> I tried to find out which commit introduced and was able to identify
>>>> revision 3908 as breaking change for me. After looking into all
>>>> changes
>>>> done in this change, I've found out that the following change
>>>> introduced my problems:
>>>> -  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
>>>> +  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];
>>>>
>>>> There is a way to overwrite this setting in your application code:
>>>> dwr.engine.setRetryIntervals([ 1, 1, 10 ]);
>>>>
>>>> It sounds really strange that just this change causes such issues.
>>>> While investigating a bit more, I have seen that the server "bombs"
>>>> the
>>>> server with a lot of "__System.checkHeartbeat.dwr" and
>>>> "ReverseAjax.dwr" requests in a short time while the server is
>>>> starting. All these requests keep the state pending and take several
>>>> seconds to more than one minute to finish. While they are pending,
>>>> dwr
>>>> issues even more requests of this type.
>>>>
>>>> Any ideas? I think you should not issue any more polling requests
>>>> while
>>>> requests are pending.
>>>>
>>>> One minor note: the setting change should also result in an update
>>>> of
>>>> the comment: "Retry every 10 seconds when offline." (you have
>>>> changed
>>>> it to 3s).
>>>>
>>>> Bye
>>>> urs
>>
Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

david@butterdev.com
 From looking at the code in Factory, it appears you are doing something
not standard.

         Builder<T> b = this.builder;
         if (b == null)
         {
             log.warn("DWR has not been initialized properly");
             return null;
         }

That log statement is only printed when this.builder is null. This
indicates that the attach method was called from outside of a DWR
thread.  We haven't experimented with initializing DWR via JavaConfig so
that could be the problem.  However, you indicated that you are seeing
this from web.xml configs as well which doesn't make a lot of sense
unless you are doing something custom with DWR initialization that you
aren't telling us about.

On 12/21/2015 06:18 PM, junkmail wrote:

> Yea that is what I was trying. I did think he asked for a stacktrace.
> Either way it is about the same what I was doing just adding a
> printStackTrace there.  Odd thing is with my test Code it does not
> even seem to be hitting that function. Since I got to that point I
> have not had time to dig into it more.  I'll definitely get back to it
> and respond with what I find so if anyone else runs into it they have
> some info. ;)
>
> Thanks.
>
>
> 21.12.2015 18:12 に David Marginian さんは書きました:
>> Sorry, I got confused with your other thread - "DWR 3.0 Build 646 and
>> Springs 4.2.3"
>>
>> Mike didn't suggest you put in stacktraces (whatever that means). He
>> suggested you download the DWR source and put some breakpoints in the
>> code to see what the call stack is when the following warning is
>> logged (per your original message) -
>> "org.directwebremoting.extend.Factory  - DWR has not been initialized
>> properly"
>>
>> Urs did not mention getting any errors in the logs so it is quite
>> possible these issues are unrelated.
>>
>> On 12/21/2015 05:32 PM, junkmail wrote:
>>> Sorry I have not responded I have not had time to work on the
>>> debuging code.  I still have the same issue as before but am trying
>>> to do as suggested putting in stack traces where you had requested.
>>> Or getting some boiled down code that can replicate the issue.    I
>>> will also try the change that Urs suggested see if it fixes the
>>> problem for me.   Urs are you saying you set this on page load in JS
>>> I assume?
>>>
>>> Thanks.
>>>
>>>
>>> 21.12.2015 08:21 に [hidden email] さんは書きました:
>>>> The reporter of this issue must have messaged me privately and
>>>> indicated the problem was with Jetty and their servlet config. I will
>>>> look up his message and post it to the list.
>>>>
>>>> I also find it strange that this is the root cause of your problem.
>>>> We have had thousands of downloads of 3.0 and no reports of something
>>>> similar. The calls in question are heartbeat calls and should be
>>>> executing extremely quickly, even if several of them queue there
>>>> should not be a problem.  If you can minimize your code and send us a
>>>> war that reproduces the problem that would be helpful.
>>>>
>>>> You are correct about the comments in engine.js.  I will update those
>>>> and also clarify the retry docs a bit
>>>> (http://directwebremoting.org/dwr/documentation/reverse-ajax/retry.html).
>>>> On 2015-12-21 00:38, [hidden email] wrote:
>>>>> I sufferred a problem which sounds almost like this problem. After
>>>>> updating to 3.0.1 (from 3.0.0-rc3-20120401), the application did not
>>>>> respond for a long time after a restart (sometimes several minutes).
>>>>>
>>>>> I tried to find out which commit introduced and was able to identify
>>>>> revision 3908 as breaking change for me. After looking into all
>>>>> changes
>>>>> done in this change, I've found out that the following change
>>>>> introduced my problems:
>>>>> -  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
>>>>> +  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];
>>>>>
>>>>> There is a way to overwrite this setting in your application code:
>>>>> dwr.engine.setRetryIntervals([ 1, 1, 10 ]);
>>>>>
>>>>> It sounds really strange that just this change causes such issues.
>>>>> While investigating a bit more, I have seen that the server
>>>>> "bombs" the
>>>>> server with a lot of "__System.checkHeartbeat.dwr" and
>>>>> "ReverseAjax.dwr" requests in a short time while the server is
>>>>> starting. All these requests keep the state pending and take several
>>>>> seconds to more than one minute to finish. While they are pending,
>>>>> dwr
>>>>> issues even more requests of this type.
>>>>>
>>>>> Any ideas? I think you should not issue any more polling requests
>>>>> while
>>>>> requests are pending.
>>>>>
>>>>> One minor note: the setting change should also result in an update of
>>>>> the comment: "Retry every 10 seconds when offline." (you have changed
>>>>> it to 3s).
>>>>>
>>>>> Bye
>>>>> urs
>>>
>

Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

uwolfer
In reply to this post by david@butterdev.com
Yes, it sounds also really strange to me. But I have builded DWR by
myself in order to find the revision which introduced this issue. And
after that, I have changed *only* the line noted below.

Some details about my setup:
* using Jetty 9.2
* ~300 DWR Java classes
* *not* using Spring or Hibernate
* DWR initialization time (including other services required by DWR):
~20-20s

I have noticed that it is possible that two (or more) ReverseAjax.dwr
are open after a restart for the same browser session (tab) which should
not be possible IMHO. I think as long as a __System.checkHeartbeat.dwr
or ReverseAjax.dwr is pending (and not failed), no other request of the
same type should be issued. You can see this behavior in the attached
screenshot (screenshot displays situation right after restart; as you
can see, there are many ReverseAjax.dwr requests which all stay there
(are re-newed) after a restart). Btw, I was not able to reproduce
multiple ReverseAjax.dwr in an older snapshot (rc3-20120401).

Bye
urs


On 2015-12-21 16:21, [hidden email] wrote:

> The reporter of this issue must have messaged me privately and
> indicated the problem was with Jetty and their servlet config.  I
> will
> look up his message and post it to the list.
>
> I also find it strange that this is the root cause of your problem.
> We have had thousands of downloads of 3.0 and no reports of something
> similar. The calls in question are heartbeat calls and should be
> executing extremely quickly, even if several of them queue there
> should not be a problem.  If you can minimize your code and send us a
> war that reproduces the problem that would be helpful.
>
> You are correct about the comments in engine.js.  I will update those
> and also clarify the retry docs a bit
>
> (http://directwebremoting.org/dwr/documentation/reverse-ajax/retry.html).
>
>
> On 2015-12-21 00:38, [hidden email] wrote:
>> I sufferred a problem which sounds almost like this problem. After
>> updating to 3.0.1 (from 3.0.0-rc3-20120401), the application did not
>> respond for a long time after a restart (sometimes several minutes).
>> I tried to find out which commit introduced and was able to identify
>> revision 3908 as breaking change for me. After looking into all
>> changes
>> done in this change, I've found out that the following change
>> introduced my problems:
>> -  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
>> +  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];
>> There is a way to overwrite this setting in your application code:
>> dwr.engine.setRetryIntervals([ 1, 1, 10 ]);
>> It sounds really strange that just this change causes such issues.
>> While investigating a bit more, I have seen that the server "bombs"
>> the
>> server with a lot of "__System.checkHeartbeat.dwr" and
>> "ReverseAjax.dwr" requests in a short time while the server is
>> starting. All these requests keep the state pending and take several
>> seconds to more than one minute to finish. While they are pending,
>> dwr
>> issues even more requests of this type.
>> Any ideas? I think you should not issue any more polling requests
>> while
>> requests are pending.
>> One minor note: the setting change should also result in an update
>> of
>> the comment: "Retry every 10 seconds when offline." (you have
>> changed
>> it to 3s).
>> Bye
>> urs
--
------------------------------------------------------------------------------
Tocco AG, technology meets spirit, Riedtlistrasse 27, CH-8006 Zürich
Tel. +41 (0)44 388 60 00

[hidden email], http://www.tocco.ch
------------------------------------------------------------------------------

dwr_restart.png (164K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

uwolfer
In reply to this post by zooppoop
On 2015-12-22 01:32, junkmail wrote:
> Sorry I have not responded I have not had time to work on the
> debuging code.  I still have the same issue as before but am trying
> to
> do as suggested putting in stack traces where you had requested.  Or
> getting some boiled down code that can replicate the issue.    I will
> also try the change that Urs suggested see if it fixes the problem
> for
> me.   Urs are you saying you set this on page load in JS I assume?

Yes, you can add this JS line right after the DWR JS include. It would
be very interested if this fixes you issues also.

Bye
urs


>
> Thanks.
>
>
> 21.12.2015 08:21 に [hidden email] さんは書きました:
>> The reporter of this issue must have messaged me privately and
>> indicated the problem was with Jetty and their servlet config.  I
>> will
>> look up his message and post it to the list.
>> I also find it strange that this is the root cause of your problem.
>> We have had thousands of downloads of 3.0 and no reports of
>> something
>> similar. The calls in question are heartbeat calls and should be
>> executing extremely quickly, even if several of them queue there
>> should not be a problem.  If you can minimize your code and send us
>> a
>> war that reproduces the problem that would be helpful.
>> You are correct about the comments in engine.js.  I will update
>> those
>> and also clarify the retry docs a bit
>>
>> (http://directwebremoting.org/dwr/documentation/reverse-ajax/retry.html).
>>
>> On 2015-12-21 00:38, [hidden email] wrote:
>>> I sufferred a problem which sounds almost like this problem. After
>>> updating to 3.0.1 (from 3.0.0-rc3-20120401), the application did
>>> not
>>> respond for a long time after a restart (sometimes several
>>> minutes).
>>> I tried to find out which commit introduced and was able to
>>> identify
>>> revision 3908 as breaking change for me. After looking into all
>>> changes
>>> done in this change, I've found out that the following change
>>> introduced my problems:
>>> -  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
>>> +  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];
>>> There is a way to overwrite this setting in your application code:
>>> dwr.engine.setRetryIntervals([ 1, 1, 10 ]);
>>> It sounds really strange that just this change causes such issues.
>>> While investigating a bit more, I have seen that the server "bombs"
>>> the
>>> server with a lot of "__System.checkHeartbeat.dwr" and
>>> "ReverseAjax.dwr" requests in a short time while the server is
>>> starting. All these requests keep the state pending and take
>>> several
>>> seconds to more than one minute to finish. While they are pending,
>>> dwr
>>> issues even more requests of this type.
>>> Any ideas? I think you should not issue any more polling requests
>>> while
>>> requests are pending.
>>> One minor note: the setting change should also result in an update
>>> of
>>> the comment: "Retry every 10 seconds when offline." (you have
>>> changed
>>> it to 3s).
>>> Bye
>>> urs
Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

david@butterdev.com
In reply to this post by uwolfer
When in the attached graphic are you restarting the server?  You have a
bunch of 200's and then _System.checkHeartbeat.dwr fails.  I would
assume the server was taken down right before then.  If that is the case
those pending ReverseAjax.dwr calls were there before the server went
down unless you cleared your console.


On 2015-12-22 00:52, Urs Wolfer wrote:

> Yes, it sounds also really strange to me. But I have builded DWR by
> myself in order to find the revision which introduced this issue. And
> after that, I have changed *only* the line noted below.
>
> Some details about my setup:
> * using Jetty 9.2
> * ~300 DWR Java classes
> * *not* using Spring or Hibernate
> * DWR initialization time (including other services required by DWR):
> ~20-20s
>
> I have noticed that it is possible that two (or more) ReverseAjax.dwr
> are open after a restart for the same browser session (tab) which
> should not be possible IMHO. I think as long as a
> __System.checkHeartbeat.dwr or ReverseAjax.dwr is pending (and not
> failed), no other request of the same type should be issued. You can
> see this behavior in the attached screenshot (screenshot displays
> situation right after restart; as you can see, there are many
> ReverseAjax.dwr requests which all stay there (are re-newed) after a
> restart). Btw, I was not able to reproduce multiple ReverseAjax.dwr in
> an older snapshot (rc3-20120401).
>
> Bye
> urs
>
>
> On 2015-12-21 16:21, [hidden email] wrote:
>> The reporter of this issue must have messaged me privately and
>> indicated the problem was with Jetty and their servlet config.  I will
>> look up his message and post it to the list.
>>
>> I also find it strange that this is the root cause of your problem.
>> We have had thousands of downloads of 3.0 and no reports of something
>> similar. The calls in question are heartbeat calls and should be
>> executing extremely quickly, even if several of them queue there
>> should not be a problem.  If you can minimize your code and send us a
>> war that reproduces the problem that would be helpful.
>>
>> You are correct about the comments in engine.js.  I will update those
>> and also clarify the retry docs a bit
>>
>> (http://directwebremoting.org/dwr/documentation/reverse-ajax/retry.html).
>>
>>
>> On 2015-12-21 00:38, [hidden email] wrote:
>>> I sufferred a problem which sounds almost like this problem. After
>>> updating to 3.0.1 (from 3.0.0-rc3-20120401), the application did not
>>> respond for a long time after a restart (sometimes several minutes).
>>> I tried to find out which commit introduced and was able to identify
>>> revision 3908 as breaking change for me. After looking into all
>>> changes
>>> done in this change, I've found out that the following change
>>> introduced my problems:
>>> -  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
>>> +  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];
>>> There is a way to overwrite this setting in your application code:
>>> dwr.engine.setRetryIntervals([ 1, 1, 10 ]);
>>> It sounds really strange that just this change causes such issues.
>>> While investigating a bit more, I have seen that the server "bombs"
>>> the
>>> server with a lot of "__System.checkHeartbeat.dwr" and
>>> "ReverseAjax.dwr" requests in a short time while the server is
>>> starting. All these requests keep the state pending and take several
>>> seconds to more than one minute to finish. While they are pending,
>>> dwr
>>> issues even more requests of this type.
>>> Any ideas? I think you should not issue any more polling requests
>>> while
>>> requests are pending.
>>> One minor note: the setting change should also result in an update of
>>> the comment: "Retry every 10 seconds when offline." (you have changed
>>> it to 3s).
>>> Bye
>>> urs
Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

uwolfer
On 2015-12-22 15:21, [hidden email] wrote:
> When in the attached graphic are you restarting the server?  You have
> a bunch of 200's and then _System.checkHeartbeat.dwr fails.  I would
> assume the server was taken down right before then.  If that is the
> case those pending ReverseAjax.dwr calls were there before the server
> went down unless you cleared your console.

The first red request (first visible request in the graphic) is the
first request right after shutdown of the application. Then I have
restarted the application and a few requests failed until Jetty was
accepting requests. All pending ReverseAjax.dwr were issued after the
restart. After every pending  _System.checkHeartbeat.dwr which got a
success-response, a ReverseAjax.dwr is started.

Bye
urs

> On 2015-12-22 00:52, Urs Wolfer wrote:
>> Yes, it sounds also really strange to me. But I have builded DWR by
>> myself in order to find the revision which introduced this issue.
>> And
>> after that, I have changed *only* the line noted below.
>> Some details about my setup:
>> * using Jetty 9.2
>> * ~300 DWR Java classes
>> * *not* using Spring or Hibernate
>> * DWR initialization time (including other services required by
>> DWR): ~20-20s
>> I have noticed that it is possible that two (or more)
>> ReverseAjax.dwr
>> are open after a restart for the same browser session (tab) which
>> should not be possible IMHO. I think as long as a
>> __System.checkHeartbeat.dwr or ReverseAjax.dwr is pending (and not
>> failed), no other request of the same type should be issued. You can
>> see this behavior in the attached screenshot (screenshot displays
>> situation right after restart; as you can see, there are many
>> ReverseAjax.dwr requests which all stay there (are re-newed) after a
>> restart). Btw, I was not able to reproduce multiple ReverseAjax.dwr
>> in
>> an older snapshot (rc3-20120401).
>> Bye
>> urs
>>
>> On 2015-12-21 16:21, [hidden email] wrote:
>>> The reporter of this issue must have messaged me privately and
>>> indicated the problem was with Jetty and their servlet config.  I
>>> will
>>> look up his message and post it to the list.
>>> I also find it strange that this is the root cause of your problem.
>>> We have had thousands of downloads of 3.0 and no reports of
>>> something
>>> similar. The calls in question are heartbeat calls and should be
>>> executing extremely quickly, even if several of them queue there
>>> should not be a problem.  If you can minimize your code and send us
>>> a
>>> war that reproduces the problem that would be helpful.
>>> You are correct about the comments in engine.js.  I will update
>>> those
>>> and also clarify the retry docs a bit
>>>
>>> (http://directwebremoting.org/dwr/documentation/reverse-ajax/retry.html).
>>>
>>> On 2015-12-21 00:38, [hidden email] wrote:
>>>> I sufferred a problem which sounds almost like this problem. After
>>>> updating to 3.0.1 (from 3.0.0-rc3-20120401), the application did
>>>> not
>>>> respond for a long time after a restart (sometimes several
>>>> minutes).
>>>> I tried to find out which commit introduced and was able to
>>>> identify
>>>> revision 3908 as breaking change for me. After looking into all
>>>> changes
>>>> done in this change, I've found out that the following change
>>>> introduced my problems:
>>>> -  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
>>>> +  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];
>>>> There is a way to overwrite this setting in your application code:
>>>> dwr.engine.setRetryIntervals([ 1, 1, 10 ]);
>>>> It sounds really strange that just this change causes such issues.
>>>> While investigating a bit more, I have seen that the server
>>>> "bombs" the
>>>> server with a lot of "__System.checkHeartbeat.dwr" and
>>>> "ReverseAjax.dwr" requests in a short time while the server is
>>>> starting. All these requests keep the state pending and take
>>>> several
>>>> seconds to more than one minute to finish. While they are pending,
>>>> dwr
>>>> issues even more requests of this type.
>>>> Any ideas? I think you should not issue any more polling requests
>>>> while
>>>> requests are pending.
>>>> One minor note: the setting change should also result in an update
>>>> of
>>>> the comment: "Retry every 10 seconds when offline." (you have
>>>> changed
>>>> it to 3s).
>>>> Bye
>>>> urs
Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

david@butterdev.com
In reply to this post by uwolfer
A few things don't quite look right in your screenshot.  First, as you
mentioned all those pending reverse ajax requests, I don't see that when
I run tests locally.  Also, the heartbeat calls after that are taking a
very long time (15 + seconds), that is definitely not right.

What does your reverse ajax configuration look like?  What happens if
you kill the browser window and start a new one after restart?  Do you
see the same behavior?  Any difference in Firefox?

On 2015-12-22 00:52, Urs Wolfer wrote:

> Yes, it sounds also really strange to me. But I have builded DWR by
> myself in order to find the revision which introduced this issue. And
> after that, I have changed *only* the line noted below.
>
> Some details about my setup:
> * using Jetty 9.2
> * ~300 DWR Java classes
> * *not* using Spring or Hibernate
> * DWR initialization time (including other services required by DWR):
> ~20-20s
>
> I have noticed that it is possible that two (or more) ReverseAjax.dwr
> are open after a restart for the same browser session (tab) which
> should not be possible IMHO. I think as long as a
> __System.checkHeartbeat.dwr or ReverseAjax.dwr is pending (and not
> failed), no other request of the same type should be issued. You can
> see this behavior in the attached screenshot (screenshot displays
> situation right after restart; as you can see, there are many
> ReverseAjax.dwr requests which all stay there (are re-newed) after a
> restart). Btw, I was not able to reproduce multiple ReverseAjax.dwr in
> an older snapshot (rc3-20120401).
>
> Bye
> urs
>
>
> On 2015-12-21 16:21, [hidden email] wrote:
>> The reporter of this issue must have messaged me privately and
>> indicated the problem was with Jetty and their servlet config.  I will
>> look up his message and post it to the list.
>>
>> I also find it strange that this is the root cause of your problem.
>> We have had thousands of downloads of 3.0 and no reports of something
>> similar. The calls in question are heartbeat calls and should be
>> executing extremely quickly, even if several of them queue there
>> should not be a problem.  If you can minimize your code and send us a
>> war that reproduces the problem that would be helpful.
>>
>> You are correct about the comments in engine.js.  I will update those
>> and also clarify the retry docs a bit
>>
>> (http://directwebremoting.org/dwr/documentation/reverse-ajax/retry.html).
>>
>>
>> On 2015-12-21 00:38, [hidden email] wrote:
>>> I sufferred a problem which sounds almost like this problem. After
>>> updating to 3.0.1 (from 3.0.0-rc3-20120401), the application did not
>>> respond for a long time after a restart (sometimes several minutes).
>>> I tried to find out which commit introduced and was able to identify
>>> revision 3908 as breaking change for me. After looking into all
>>> changes
>>> done in this change, I've found out that the following change
>>> introduced my problems:
>>> -  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
>>> +  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];
>>> There is a way to overwrite this setting in your application code:
>>> dwr.engine.setRetryIntervals([ 1, 1, 10 ]);
>>> It sounds really strange that just this change causes such issues.
>>> While investigating a bit more, I have seen that the server "bombs"
>>> the
>>> server with a lot of "__System.checkHeartbeat.dwr" and
>>> "ReverseAjax.dwr" requests in a short time while the server is
>>> starting. All these requests keep the state pending and take several
>>> seconds to more than one minute to finish. While they are pending,
>>> dwr
>>> issues even more requests of this type.
>>> Any ideas? I think you should not issue any more polling requests
>>> while
>>> requests are pending.
>>> One minor note: the setting change should also result in an update of
>>> the comment: "Retry every 10 seconds when offline." (you have changed
>>> it to 3s).
>>> Bye
>>> urs
Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

david@butterdev.com
In reply to this post by uwolfer
The more I look at it that graphic really is odd.  After the first
heartbeat call succeeds we clear the interval and no more requests
should be made.  I noticed you have 8 heartbeat 200 heartbeat requests
after the server comes up and 8 ReverseAjax.dwr calls.  Also, the calls
are taking a super long time, that call should be taking a millisecond.  
It is like you have 8 instances of engine.js running or something.  You
mentioned you have 300+ classes exposed to DWR, if you reduce that to
say 10, do you see the same behavior?


On 2015-12-22 07:51, Urs Wolfer wrote:

> On 2015-12-22 15:21, [hidden email] wrote:
>> When in the attached graphic are you restarting the server?  You have
>> a bunch of 200's and then _System.checkHeartbeat.dwr fails.  I would
>> assume the server was taken down right before then.  If that is the
>> case those pending ReverseAjax.dwr calls were there before the server
>> went down unless you cleared your console.
>
> The first red request (first visible request in the graphic) is the
> first request right after shutdown of the application. Then I have
> restarted the application and a few requests failed until Jetty was
> accepting requests. All pending ReverseAjax.dwr were issued after the
> restart. After every pending  _System.checkHeartbeat.dwr which got a
> success-response, a ReverseAjax.dwr is started.
>
> Bye
> urs
>
>> On 2015-12-22 00:52, Urs Wolfer wrote:
>>> Yes, it sounds also really strange to me. But I have builded DWR by
>>> myself in order to find the revision which introduced this issue. And
>>> after that, I have changed *only* the line noted below.
>>> Some details about my setup:
>>> * using Jetty 9.2
>>> * ~300 DWR Java classes
>>> * *not* using Spring or Hibernate
>>> * DWR initialization time (including other services required by DWR):
>>> ~20-20s
>>> I have noticed that it is possible that two (or more) ReverseAjax.dwr
>>> are open after a restart for the same browser session (tab) which
>>> should not be possible IMHO. I think as long as a
>>> __System.checkHeartbeat.dwr or ReverseAjax.dwr is pending (and not
>>> failed), no other request of the same type should be issued. You can
>>> see this behavior in the attached screenshot (screenshot displays
>>> situation right after restart; as you can see, there are many
>>> ReverseAjax.dwr requests which all stay there (are re-newed) after a
>>> restart). Btw, I was not able to reproduce multiple ReverseAjax.dwr
>>> in
>>> an older snapshot (rc3-20120401).
>>> Bye
>>> urs
>>>
>>> On 2015-12-21 16:21, [hidden email] wrote:
>>>> The reporter of this issue must have messaged me privately and
>>>> indicated the problem was with Jetty and their servlet config.  I
>>>> will
>>>> look up his message and post it to the list.
>>>> I also find it strange that this is the root cause of your problem.
>>>> We have had thousands of downloads of 3.0 and no reports of
>>>> something
>>>> similar. The calls in question are heartbeat calls and should be
>>>> executing extremely quickly, even if several of them queue there
>>>> should not be a problem.  If you can minimize your code and send us
>>>> a
>>>> war that reproduces the problem that would be helpful.
>>>> You are correct about the comments in engine.js.  I will update
>>>> those
>>>> and also clarify the retry docs a bit
>>>>
>>>> (http://directwebremoting.org/dwr/documentation/reverse-ajax/retry.html).
>>>>
>>>> On 2015-12-21 00:38, [hidden email] wrote:
>>>>> I sufferred a problem which sounds almost like this problem. After
>>>>> updating to 3.0.1 (from 3.0.0-rc3-20120401), the application did
>>>>> not
>>>>> respond for a long time after a restart (sometimes several
>>>>> minutes).
>>>>> I tried to find out which commit introduced and was able to
>>>>> identify
>>>>> revision 3908 as breaking change for me. After looking into all
>>>>> changes
>>>>> done in this change, I've found out that the following change
>>>>> introduced my problems:
>>>>> -  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
>>>>> +  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];
>>>>> There is a way to overwrite this setting in your application code:
>>>>> dwr.engine.setRetryIntervals([ 1, 1, 10 ]);
>>>>> It sounds really strange that just this change causes such issues.
>>>>> While investigating a bit more, I have seen that the server "bombs"
>>>>> the
>>>>> server with a lot of "__System.checkHeartbeat.dwr" and
>>>>> "ReverseAjax.dwr" requests in a short time while the server is
>>>>> starting. All these requests keep the state pending and take
>>>>> several
>>>>> seconds to more than one minute to finish. While they are pending,
>>>>> dwr
>>>>> issues even more requests of this type.
>>>>> Any ideas? I think you should not issue any more polling requests
>>>>> while
>>>>> requests are pending.
>>>>> One minor note: the setting change should also result in an update
>>>>> of
>>>>> the comment: "Retry every 10 seconds when offline." (you have
>>>>> changed
>>>>> it to 3s).
>>>>> Bye
>>>>> urs
Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

uwolfer
I do not have 8 engines.js included. 8 is the number of retries which
were tried while the application stared up. It's not always 8, but
around that (since application startup takes around 30s; 8 * 3s
(interval)).

I can reproduce the same issue in Firefox as well.

__System.checkHeartbeat.dwr normally takes just a few ms, but while
restarting, these requests are queued up until DWR is ready (and as I
wrote, DWR takes around 20-30s). I have not tried with less Java
services yet, but I assume that it is faster to start up.

I think you should stop requesting __System.checkHeartbeat.dwr when the
request not has returned yet (failure or success). Otherwise every
__System.checkHeartbeat.dwr which finally returns opens a
reverse-connection.

I have no special reverse-ajax configuration (just activated it on
server and client).

You probably could try to delay DWR startup for about 30s. I have not
tried to reproduce it in a standalone way yet, but it should be quite
easy when you add a breakpoint while restarting.

Bye
urs


On 2015-12-22 16:05, [hidden email] wrote:

> The more I look at it that graphic really is odd.  After the first
> heartbeat call succeeds we clear the interval and no more requests
> should be made.  I noticed you have 8 heartbeat 200 heartbeat
> requests
> after the server comes up and 8 ReverseAjax.dwr calls.  Also, the
> calls are taking a super long time, that call should be taking a
> millisecond.  It is like you have 8 instances of engine.js running or
> something.  You mentioned you have 300+ classes exposed to DWR, if
> you
> reduce that to say 10, do you see the same behavior?
>
>
> On 2015-12-22 07:51, Urs Wolfer wrote:
>> On 2015-12-22 15:21, [hidden email] wrote:
>>> When in the attached graphic are you restarting the server?  You
>>> have
>>> a bunch of 200's and then _System.checkHeartbeat.dwr fails.  I
>>> would
>>> assume the server was taken down right before then.  If that is the
>>> case those pending ReverseAjax.dwr calls were there before the
>>> server
>>> went down unless you cleared your console.
>> The first red request (first visible request in the graphic) is the
>> first request right after shutdown of the application. Then I have
>> restarted the application and a few requests failed until Jetty was
>> accepting requests. All pending ReverseAjax.dwr were issued after
>> the
>> restart. After every pending  _System.checkHeartbeat.dwr which got a
>> success-response, a ReverseAjax.dwr is started.
>> Bye
>> urs
>>
>>> On 2015-12-22 00:52, Urs Wolfer wrote:
>>>> Yes, it sounds also really strange to me. But I have builded DWR
>>>> by
>>>> myself in order to find the revision which introduced this issue.
>>>> And
>>>> after that, I have changed *only* the line noted below.
>>>> Some details about my setup:
>>>> * using Jetty 9.2
>>>> * ~300 DWR Java classes
>>>> * *not* using Spring or Hibernate
>>>> * DWR initialization time (including other services required by
>>>> DWR): ~20-20s
>>>> I have noticed that it is possible that two (or more)
>>>> ReverseAjax.dwr
>>>> are open after a restart for the same browser session (tab) which
>>>> should not be possible IMHO. I think as long as a
>>>> __System.checkHeartbeat.dwr or ReverseAjax.dwr is pending (and not
>>>> failed), no other request of the same type should be issued. You
>>>> can
>>>> see this behavior in the attached screenshot (screenshot displays
>>>> situation right after restart; as you can see, there are many
>>>> ReverseAjax.dwr requests which all stay there (are re-newed) after
>>>> a
>>>> restart). Btw, I was not able to reproduce multiple
>>>> ReverseAjax.dwr in
>>>> an older snapshot (rc3-20120401).
>>>> Bye
>>>> urs
>>>> On 2015-12-21 16:21, [hidden email] wrote:
>>>>> The reporter of this issue must have messaged me privately and
>>>>> indicated the problem was with Jetty and their servlet config.  I
>>>>> will
>>>>> look up his message and post it to the list.
>>>>> I also find it strange that this is the root cause of your
>>>>> problem.
>>>>> We have had thousands of downloads of 3.0 and no reports of
>>>>> something
>>>>> similar. The calls in question are heartbeat calls and should be
>>>>> executing extremely quickly, even if several of them queue there
>>>>> should not be a problem.  If you can minimize your code and send
>>>>> us a
>>>>> war that reproduces the problem that would be helpful.
>>>>> You are correct about the comments in engine.js.  I will update
>>>>> those
>>>>> and also clarify the retry docs a bit
>>>>>
>>>>> (http://directwebremoting.org/dwr/documentation/reverse-ajax/retry.html).
>>>>> On 2015-12-21 00:38, [hidden email] wrote:
>>>>>> I sufferred a problem which sounds almost like this problem.
>>>>>> After
>>>>>> updating to 3.0.1 (from 3.0.0-rc3-20120401), the application did
>>>>>> not
>>>>>> respond for a long time after a restart (sometimes several
>>>>>> minutes).
>>>>>> I tried to find out which commit introduced and was able to
>>>>>> identify
>>>>>> revision 3908 as breaking change for me. After looking into all
>>>>>> changes
>>>>>> done in this change, I've found out that the following change
>>>>>> introduced my problems:
>>>>>> -  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
>>>>>> +  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];
>>>>>> There is a way to overwrite this setting in your application
>>>>>> code:
>>>>>> dwr.engine.setRetryIntervals([ 1, 1, 10 ]);
>>>>>> It sounds really strange that just this change causes such
>>>>>> issues.
>>>>>> While investigating a bit more, I have seen that the server
>>>>>> "bombs" the
>>>>>> server with a lot of "__System.checkHeartbeat.dwr" and
>>>>>> "ReverseAjax.dwr" requests in a short time while the server is
>>>>>> starting. All these requests keep the state pending and take
>>>>>> several
>>>>>> seconds to more than one minute to finish. While they are
>>>>>> pending, dwr
>>>>>> issues even more requests of this type.
>>>>>> Any ideas? I think you should not issue any more polling
>>>>>> requests while
>>>>>> requests are pending.
>>>>>> One minor note: the setting change should also result in an
>>>>>> update of
>>>>>> the comment: "Retry every 10 seconds when offline." (you have
>>>>>> changed
>>>>>> it to 3s).
>>>>>> Bye
>>>>>> urs
Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

david@butterdev.com
Ok, I think I have a good understanding of what is going on here.  I
have created an issue, thanks for the help.
https://directwebremoting.atlassian.net/browse/DWR-654

Perhaps, a quick solution to this is to set a short timeout on the
heartbeat calls.  I will investigate that and other options.


On 2015-12-22 09:09, Urs Wolfer wrote:

> I do not have 8 engines.js included. 8 is the number of retries which
> were tried while the application stared up. It's not always 8, but
> around that (since application startup takes around 30s; 8 * 3s
> (interval)).
>
> I can reproduce the same issue in Firefox as well.
>
> __System.checkHeartbeat.dwr normally takes just a few ms, but while
> restarting, these requests are queued up until DWR is ready (and as I
> wrote, DWR takes around 20-30s). I have not tried with less Java
> services yet, but I assume that it is faster to start up.
>
> I think you should stop requesting __System.checkHeartbeat.dwr when
> the request not has returned yet (failure or success). Otherwise every
> __System.checkHeartbeat.dwr which finally returns opens a
> reverse-connection.
>
> I have no special reverse-ajax configuration (just activated it on
> server and client).
>
> You probably could try to delay DWR startup for about 30s. I have not
> tried to reproduce it in a standalone way yet, but it should be quite
> easy when you add a breakpoint while restarting.
>
> Bye
> urs
>
>
> On 2015-12-22 16:05, [hidden email] wrote:
>> The more I look at it that graphic really is odd.  After the first
>> heartbeat call succeeds we clear the interval and no more requests
>> should be made.  I noticed you have 8 heartbeat 200 heartbeat requests
>> after the server comes up and 8 ReverseAjax.dwr calls.  Also, the
>> calls are taking a super long time, that call should be taking a
>> millisecond.  It is like you have 8 instances of engine.js running or
>> something.  You mentioned you have 300+ classes exposed to DWR, if you
>> reduce that to say 10, do you see the same behavior?
>>
>>
>> On 2015-12-22 07:51, Urs Wolfer wrote:
>>> On 2015-12-22 15:21, [hidden email] wrote:
>>>> When in the attached graphic are you restarting the server?  You
>>>> have
>>>> a bunch of 200's and then _System.checkHeartbeat.dwr fails.  I would
>>>> assume the server was taken down right before then.  If that is the
>>>> case those pending ReverseAjax.dwr calls were there before the
>>>> server
>>>> went down unless you cleared your console.
>>> The first red request (first visible request in the graphic) is the
>>> first request right after shutdown of the application. Then I have
>>> restarted the application and a few requests failed until Jetty was
>>> accepting requests. All pending ReverseAjax.dwr were issued after the
>>> restart. After every pending  _System.checkHeartbeat.dwr which got a
>>> success-response, a ReverseAjax.dwr is started.
>>> Bye
>>> urs
>>>
>>>> On 2015-12-22 00:52, Urs Wolfer wrote:
>>>>> Yes, it sounds also really strange to me. But I have builded DWR by
>>>>> myself in order to find the revision which introduced this issue.
>>>>> And
>>>>> after that, I have changed *only* the line noted below.
>>>>> Some details about my setup:
>>>>> * using Jetty 9.2
>>>>> * ~300 DWR Java classes
>>>>> * *not* using Spring or Hibernate
>>>>> * DWR initialization time (including other services required by
>>>>> DWR): ~20-20s
>>>>> I have noticed that it is possible that two (or more)
>>>>> ReverseAjax.dwr
>>>>> are open after a restart for the same browser session (tab) which
>>>>> should not be possible IMHO. I think as long as a
>>>>> __System.checkHeartbeat.dwr or ReverseAjax.dwr is pending (and not
>>>>> failed), no other request of the same type should be issued. You
>>>>> can
>>>>> see this behavior in the attached screenshot (screenshot displays
>>>>> situation right after restart; as you can see, there are many
>>>>> ReverseAjax.dwr requests which all stay there (are re-newed) after
>>>>> a
>>>>> restart). Btw, I was not able to reproduce multiple ReverseAjax.dwr
>>>>> in
>>>>> an older snapshot (rc3-20120401).
>>>>> Bye
>>>>> urs
>>>>> On 2015-12-21 16:21, [hidden email] wrote:
>>>>>> The reporter of this issue must have messaged me privately and
>>>>>> indicated the problem was with Jetty and their servlet config.  I
>>>>>> will
>>>>>> look up his message and post it to the list.
>>>>>> I also find it strange that this is the root cause of your
>>>>>> problem.
>>>>>> We have had thousands of downloads of 3.0 and no reports of
>>>>>> something
>>>>>> similar. The calls in question are heartbeat calls and should be
>>>>>> executing extremely quickly, even if several of them queue there
>>>>>> should not be a problem.  If you can minimize your code and send
>>>>>> us a
>>>>>> war that reproduces the problem that would be helpful.
>>>>>> You are correct about the comments in engine.js.  I will update
>>>>>> those
>>>>>> and also clarify the retry docs a bit
>>>>>>
>>>>>> (http://directwebremoting.org/dwr/documentation/reverse-ajax/retry.html).
>>>>>> On 2015-12-21 00:38, [hidden email] wrote:
>>>>>>> I sufferred a problem which sounds almost like this problem.
>>>>>>> After
>>>>>>> updating to 3.0.1 (from 3.0.0-rc3-20120401), the application did
>>>>>>> not
>>>>>>> respond for a long time after a restart (sometimes several
>>>>>>> minutes).
>>>>>>> I tried to find out which commit introduced and was able to
>>>>>>> identify
>>>>>>> revision 3908 as breaking change for me. After looking into all
>>>>>>> changes
>>>>>>> done in this change, I've found out that the following change
>>>>>>> introduced my problems:
>>>>>>> -  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
>>>>>>> +  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];
>>>>>>> There is a way to overwrite this setting in your application
>>>>>>> code:
>>>>>>> dwr.engine.setRetryIntervals([ 1, 1, 10 ]);
>>>>>>> It sounds really strange that just this change causes such
>>>>>>> issues.
>>>>>>> While investigating a bit more, I have seen that the server
>>>>>>> "bombs" the
>>>>>>> server with a lot of "__System.checkHeartbeat.dwr" and
>>>>>>> "ReverseAjax.dwr" requests in a short time while the server is
>>>>>>> starting. All these requests keep the state pending and take
>>>>>>> several
>>>>>>> seconds to more than one minute to finish. While they are
>>>>>>> pending, dwr
>>>>>>> issues even more requests of this type.
>>>>>>> Any ideas? I think you should not issue any more polling requests
>>>>>>> while
>>>>>>> requests are pending.
>>>>>>> One minor note: the setting change should also result in an
>>>>>>> update of
>>>>>>> the comment: "Retry every 10 seconds when offline." (you have
>>>>>>> changed
>>>>>>> it to 3s).
>>>>>>> Bye
>>>>>>> urs
Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

david@butterdev.com
Urs,
I would be curious for you to try -
http://ci.directwebremoting.org/bamboo/browse/DWRTRUNK-ALL-655/artifact.
  I added a timeout for the heartbeat calls so they fail fast.  I haven't
had time to test it yet but I think it may resolve your issues.

On 2015-12-22 10:13, [hidden email] wrote:

> Ok, I think I have a good understanding of what is going on here.  I
> have created an issue, thanks for the help.
> https://directwebremoting.atlassian.net/browse/DWR-654
>
> Perhaps, a quick solution to this is to set a short timeout on the
> heartbeat calls.  I will investigate that and other options.
>
>
> On 2015-12-22 09:09, Urs Wolfer wrote:
>> I do not have 8 engines.js included. 8 is the number of retries which
>> were tried while the application stared up. It's not always 8, but
>> around that (since application startup takes around 30s; 8 * 3s
>> (interval)).
>>
>> I can reproduce the same issue in Firefox as well.
>>
>> __System.checkHeartbeat.dwr normally takes just a few ms, but while
>> restarting, these requests are queued up until DWR is ready (and as I
>> wrote, DWR takes around 20-30s). I have not tried with less Java
>> services yet, but I assume that it is faster to start up.
>>
>> I think you should stop requesting __System.checkHeartbeat.dwr when
>> the request not has returned yet (failure or success). Otherwise every
>> __System.checkHeartbeat.dwr which finally returns opens a
>> reverse-connection.
>>
>> I have no special reverse-ajax configuration (just activated it on
>> server and client).
>>
>> You probably could try to delay DWR startup for about 30s. I have not
>> tried to reproduce it in a standalone way yet, but it should be quite
>> easy when you add a breakpoint while restarting.
>>
>> Bye
>> urs
>>
>>
>> On 2015-12-22 16:05, [hidden email] wrote:
>>> The more I look at it that graphic really is odd.  After the first
>>> heartbeat call succeeds we clear the interval and no more requests
>>> should be made.  I noticed you have 8 heartbeat 200 heartbeat
>>> requests
>>> after the server comes up and 8 ReverseAjax.dwr calls.  Also, the
>>> calls are taking a super long time, that call should be taking a
>>> millisecond.  It is like you have 8 instances of engine.js running or
>>> something.  You mentioned you have 300+ classes exposed to DWR, if
>>> you
>>> reduce that to say 10, do you see the same behavior?
>>>
>>>
>>> On 2015-12-22 07:51, Urs Wolfer wrote:
>>>> On 2015-12-22 15:21, [hidden email] wrote:
>>>>> When in the attached graphic are you restarting the server?  You
>>>>> have
>>>>> a bunch of 200's and then _System.checkHeartbeat.dwr fails.  I
>>>>> would
>>>>> assume the server was taken down right before then.  If that is the
>>>>> case those pending ReverseAjax.dwr calls were there before the
>>>>> server
>>>>> went down unless you cleared your console.
>>>> The first red request (first visible request in the graphic) is the
>>>> first request right after shutdown of the application. Then I have
>>>> restarted the application and a few requests failed until Jetty was
>>>> accepting requests. All pending ReverseAjax.dwr were issued after
>>>> the
>>>> restart. After every pending  _System.checkHeartbeat.dwr which got a
>>>> success-response, a ReverseAjax.dwr is started.
>>>> Bye
>>>> urs
>>>>
>>>>> On 2015-12-22 00:52, Urs Wolfer wrote:
>>>>>> Yes, it sounds also really strange to me. But I have builded DWR
>>>>>> by
>>>>>> myself in order to find the revision which introduced this issue.
>>>>>> And
>>>>>> after that, I have changed *only* the line noted below.
>>>>>> Some details about my setup:
>>>>>> * using Jetty 9.2
>>>>>> * ~300 DWR Java classes
>>>>>> * *not* using Spring or Hibernate
>>>>>> * DWR initialization time (including other services required by
>>>>>> DWR): ~20-20s
>>>>>> I have noticed that it is possible that two (or more)
>>>>>> ReverseAjax.dwr
>>>>>> are open after a restart for the same browser session (tab) which
>>>>>> should not be possible IMHO. I think as long as a
>>>>>> __System.checkHeartbeat.dwr or ReverseAjax.dwr is pending (and not
>>>>>> failed), no other request of the same type should be issued. You
>>>>>> can
>>>>>> see this behavior in the attached screenshot (screenshot displays
>>>>>> situation right after restart; as you can see, there are many
>>>>>> ReverseAjax.dwr requests which all stay there (are re-newed) after
>>>>>> a
>>>>>> restart). Btw, I was not able to reproduce multiple
>>>>>> ReverseAjax.dwr in
>>>>>> an older snapshot (rc3-20120401).
>>>>>> Bye
>>>>>> urs
>>>>>> On 2015-12-21 16:21, [hidden email] wrote:
>>>>>>> The reporter of this issue must have messaged me privately and
>>>>>>> indicated the problem was with Jetty and their servlet config.  I
>>>>>>> will
>>>>>>> look up his message and post it to the list.
>>>>>>> I also find it strange that this is the root cause of your
>>>>>>> problem.
>>>>>>> We have had thousands of downloads of 3.0 and no reports of
>>>>>>> something
>>>>>>> similar. The calls in question are heartbeat calls and should be
>>>>>>> executing extremely quickly, even if several of them queue there
>>>>>>> should not be a problem.  If you can minimize your code and send
>>>>>>> us a
>>>>>>> war that reproduces the problem that would be helpful.
>>>>>>> You are correct about the comments in engine.js.  I will update
>>>>>>> those
>>>>>>> and also clarify the retry docs a bit
>>>>>>>
>>>>>>> (http://directwebremoting.org/dwr/documentation/reverse-ajax/retry.html).
>>>>>>> On 2015-12-21 00:38, [hidden email] wrote:
>>>>>>>> I sufferred a problem which sounds almost like this problem.
>>>>>>>> After
>>>>>>>> updating to 3.0.1 (from 3.0.0-rc3-20120401), the application did
>>>>>>>> not
>>>>>>>> respond for a long time after a restart (sometimes several
>>>>>>>> minutes).
>>>>>>>> I tried to find out which commit introduced and was able to
>>>>>>>> identify
>>>>>>>> revision 3908 as breaking change for me. After looking into all
>>>>>>>> changes
>>>>>>>> done in this change, I've found out that the following change
>>>>>>>> introduced my problems:
>>>>>>>> -  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
>>>>>>>> +  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];
>>>>>>>> There is a way to overwrite this setting in your application
>>>>>>>> code:
>>>>>>>> dwr.engine.setRetryIntervals([ 1, 1, 10 ]);
>>>>>>>> It sounds really strange that just this change causes such
>>>>>>>> issues.
>>>>>>>> While investigating a bit more, I have seen that the server
>>>>>>>> "bombs" the
>>>>>>>> server with a lot of "__System.checkHeartbeat.dwr" and
>>>>>>>> "ReverseAjax.dwr" requests in a short time while the server is
>>>>>>>> starting. All these requests keep the state pending and take
>>>>>>>> several
>>>>>>>> seconds to more than one minute to finish. While they are
>>>>>>>> pending, dwr
>>>>>>>> issues even more requests of this type.
>>>>>>>> Any ideas? I think you should not issue any more polling
>>>>>>>> requests while
>>>>>>>> requests are pending.
>>>>>>>> One minor note: the setting change should also result in an
>>>>>>>> update of
>>>>>>>> the comment: "Retry every 10 seconds when offline." (you have
>>>>>>>> changed
>>>>>>>> it to 3s).
>>>>>>>> Bye
>>>>>>>> urs
Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

uwolfer
David, thanks for your work on it. Your fix almost fixes the issue -
but I had to move your check "dwr.engine._isHeartbeatBatch" directly
below "dwr.engine.batch.addCall". Otherwise it always returned false.
When the check is there, it works fine for me. No more than one
reverse-request is open at the same time.

Bye
urs


On 2015-12-22 19:04, [hidden email] wrote:

> Urs,
> I would be curious for you to try -
>
> http://ci.directwebremoting.org/bamboo/browse/DWRTRUNK-ALL-655/artifact.
>  I added a timeout for the heartbeat calls so they fail fast.  I
> haven't had time to test it yet but I think it may resolve your
> issues.
>
> On 2015-12-22 10:13, [hidden email] wrote:
>> Ok, I think I have a good understanding of what is going on here.  I
>> have created an issue, thanks for the help.
>> https://directwebremoting.atlassian.net/browse/DWR-654
>> Perhaps, a quick solution to this is to set a short timeout on the
>> heartbeat calls.  I will investigate that and other options.
>>
>> On 2015-12-22 09:09, Urs Wolfer wrote:
>>> I do not have 8 engines.js included. 8 is the number of retries
>>> which
>>> were tried while the application stared up. It's not always 8, but
>>> around that (since application startup takes around 30s; 8 * 3s
>>> (interval)).
>>> I can reproduce the same issue in Firefox as well.
>>> __System.checkHeartbeat.dwr normally takes just a few ms, but while
>>> restarting, these requests are queued up until DWR is ready (and as
>>> I
>>> wrote, DWR takes around 20-30s). I have not tried with less Java
>>> services yet, but I assume that it is faster to start up.
>>> I think you should stop requesting __System.checkHeartbeat.dwr when
>>> the request not has returned yet (failure or success). Otherwise
>>> every
>>> __System.checkHeartbeat.dwr which finally returns opens a
>>> reverse-connection.
>>> I have no special reverse-ajax configuration (just activated it on
>>> server and client).
>>> You probably could try to delay DWR startup for about 30s. I have
>>> not
>>> tried to reproduce it in a standalone way yet, but it should be
>>> quite
>>> easy when you add a breakpoint while restarting.
>>> Bye
>>> urs
>>>
>>> On 2015-12-22 16:05, [hidden email] wrote:
>>>> The more I look at it that graphic really is odd.  After the first
>>>> heartbeat call succeeds we clear the interval and no more requests
>>>> should be made.  I noticed you have 8 heartbeat 200 heartbeat
>>>> requests
>>>> after the server comes up and 8 ReverseAjax.dwr calls.  Also, the
>>>> calls are taking a super long time, that call should be taking a
>>>> millisecond.  It is like you have 8 instances of engine.js running
>>>> or
>>>> something.  You mentioned you have 300+ classes exposed to DWR, if
>>>> you
>>>> reduce that to say 10, do you see the same behavior?
>>>>
>>>> On 2015-12-22 07:51, Urs Wolfer wrote:
>>>>> On 2015-12-22 15:21, [hidden email] wrote:
>>>>>> When in the attached graphic are you restarting the server?  You
>>>>>> have
>>>>>> a bunch of 200's and then _System.checkHeartbeat.dwr fails.  I
>>>>>> would
>>>>>> assume the server was taken down right before then.  If that is
>>>>>> the
>>>>>> case those pending ReverseAjax.dwr calls were there before the
>>>>>> server
>>>>>> went down unless you cleared your console.
>>>>> The first red request (first visible request in the graphic) is
>>>>> the
>>>>> first request right after shutdown of the application. Then I
>>>>> have
>>>>> restarted the application and a few requests failed until Jetty
>>>>> was
>>>>> accepting requests. All pending ReverseAjax.dwr were issued after
>>>>> the
>>>>> restart. After every pending  _System.checkHeartbeat.dwr which
>>>>> got a
>>>>> success-response, a ReverseAjax.dwr is started.
>>>>> Bye
>>>>> urs
>>>>>
>>>>>> On 2015-12-22 00:52, Urs Wolfer wrote:
>>>>>>> Yes, it sounds also really strange to me. But I have builded
>>>>>>> DWR by
>>>>>>> myself in order to find the revision which introduced this
>>>>>>> issue. And
>>>>>>> after that, I have changed *only* the line noted below.
>>>>>>> Some details about my setup:
>>>>>>> * using Jetty 9.2
>>>>>>> * ~300 DWR Java classes
>>>>>>> * *not* using Spring or Hibernate
>>>>>>> * DWR initialization time (including other services required by
>>>>>>> DWR): ~20-20s
>>>>>>> I have noticed that it is possible that two (or more)
>>>>>>> ReverseAjax.dwr
>>>>>>> are open after a restart for the same browser session (tab)
>>>>>>> which
>>>>>>> should not be possible IMHO. I think as long as a
>>>>>>> __System.checkHeartbeat.dwr or ReverseAjax.dwr is pending (and
>>>>>>> not
>>>>>>> failed), no other request of the same type should be issued.
>>>>>>> You can
>>>>>>> see this behavior in the attached screenshot (screenshot
>>>>>>> displays
>>>>>>> situation right after restart; as you can see, there are many
>>>>>>> ReverseAjax.dwr requests which all stay there (are re-newed)
>>>>>>> after a
>>>>>>> restart). Btw, I was not able to reproduce multiple
>>>>>>> ReverseAjax.dwr in
>>>>>>> an older snapshot (rc3-20120401).
>>>>>>> Bye
>>>>>>> urs
>>>>>>> On 2015-12-21 16:21, [hidden email] wrote:
>>>>>>>> The reporter of this issue must have messaged me privately and
>>>>>>>> indicated the problem was with Jetty and their servlet config.
>>>>>>>> I will
>>>>>>>> look up his message and post it to the list.
>>>>>>>> I also find it strange that this is the root cause of your
>>>>>>>> problem.
>>>>>>>> We have had thousands of downloads of 3.0 and no reports of
>>>>>>>> something
>>>>>>>> similar. The calls in question are heartbeat calls and should
>>>>>>>> be
>>>>>>>> executing extremely quickly, even if several of them queue
>>>>>>>> there
>>>>>>>> should not be a problem.  If you can minimize your code and
>>>>>>>> send us a
>>>>>>>> war that reproduces the problem that would be helpful.
>>>>>>>> You are correct about the comments in engine.js.  I will
>>>>>>>> update those
>>>>>>>> and also clarify the retry docs a bit
>>>>>>>>
>>>>>>>> (http://directwebremoting.org/dwr/documentation/reverse-ajax/retry.html).
>>>>>>>> On 2015-12-21 00:38, [hidden email] wrote:
>>>>>>>>> I sufferred a problem which sounds almost like this problem.
>>>>>>>>> After
>>>>>>>>> updating to 3.0.1 (from 3.0.0-rc3-20120401), the application
>>>>>>>>> did not
>>>>>>>>> respond for a long time after a restart (sometimes several
>>>>>>>>> minutes).
>>>>>>>>> I tried to find out which commit introduced and was able to
>>>>>>>>> identify
>>>>>>>>> revision 3908 as breaking change for me. After looking into
>>>>>>>>> all changes
>>>>>>>>> done in this change, I've found out that the following change
>>>>>>>>> introduced my problems:
>>>>>>>>> -  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
>>>>>>>>> +  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];
>>>>>>>>> There is a way to overwrite this setting in your application
>>>>>>>>> code:
>>>>>>>>> dwr.engine.setRetryIntervals([ 1, 1, 10 ]);
>>>>>>>>> It sounds really strange that just this change causes such
>>>>>>>>> issues.
>>>>>>>>> While investigating a bit more, I have seen that the server
>>>>>>>>> "bombs" the
>>>>>>>>> server with a lot of "__System.checkHeartbeat.dwr" and
>>>>>>>>> "ReverseAjax.dwr" requests in a short time while the server
>>>>>>>>> is
>>>>>>>>> starting. All these requests keep the state pending and take
>>>>>>>>> several
>>>>>>>>> seconds to more than one minute to finish. While they are
>>>>>>>>> pending, dwr
>>>>>>>>> issues even more requests of this type.
>>>>>>>>> Any ideas? I think you should not issue any more polling
>>>>>>>>> requests while
>>>>>>>>> requests are pending.
>>>>>>>>> One minor note: the setting change should also result in an
>>>>>>>>> update of
>>>>>>>>> the comment: "Retry every 10 seconds when offline." (you have
>>>>>>>>> changed
>>>>>>>>> it to 3s).
>>>>>>>>> Bye
>>>>>>>>> urs
Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

david@butterdev.com
Ah, yes.  Thanks for tracking this down for us.  I have fixed this.  
Also, the suggestion you had on the Jetty cookie issue looked good.  
There is a new build out with both fixes -
http://ci.directwebremoting.org/bamboo/browse/DWRTRUNK-ALL-657/artifact.
  I am also pushing this build to Sonatype OSS repo if you use Maven.

On 2015-12-23 01:05, Urs Wolfer wrote:

> David, thanks for your work on it. Your fix almost fixes the issue -
> but I had to move your check "dwr.engine._isHeartbeatBatch" directly
> below "dwr.engine.batch.addCall". Otherwise it always returned false.
> When the check is there, it works fine for me. No more than one
> reverse-request is open at the same time.
>
> Bye
> urs
>
>
> On 2015-12-22 19:04, [hidden email] wrote:
>> Urs,
>> I would be curious for you to try -
>>
>> http://ci.directwebremoting.org/bamboo/browse/DWRTRUNK-ALL-655/artifact.
>>  I added a timeout for the heartbeat calls so they fail fast.  I
>> haven't had time to test it yet but I think it may resolve your
>> issues.
>>
>> On 2015-12-22 10:13, [hidden email] wrote:
>>> Ok, I think I have a good understanding of what is going on here.  I
>>> have created an issue, thanks for the help.
>>> https://directwebremoting.atlassian.net/browse/DWR-654
>>> Perhaps, a quick solution to this is to set a short timeout on the
>>> heartbeat calls.  I will investigate that and other options.
>>>
>>> On 2015-12-22 09:09, Urs Wolfer wrote:
>>>> I do not have 8 engines.js included. 8 is the number of retries
>>>> which
>>>> were tried while the application stared up. It's not always 8, but
>>>> around that (since application startup takes around 30s; 8 * 3s
>>>> (interval)).
>>>> I can reproduce the same issue in Firefox as well.
>>>> __System.checkHeartbeat.dwr normally takes just a few ms, but while
>>>> restarting, these requests are queued up until DWR is ready (and as
>>>> I
>>>> wrote, DWR takes around 20-30s). I have not tried with less Java
>>>> services yet, but I assume that it is faster to start up.
>>>> I think you should stop requesting __System.checkHeartbeat.dwr when
>>>> the request not has returned yet (failure or success). Otherwise
>>>> every
>>>> __System.checkHeartbeat.dwr which finally returns opens a
>>>> reverse-connection.
>>>> I have no special reverse-ajax configuration (just activated it on
>>>> server and client).
>>>> You probably could try to delay DWR startup for about 30s. I have
>>>> not
>>>> tried to reproduce it in a standalone way yet, but it should be
>>>> quite
>>>> easy when you add a breakpoint while restarting.
>>>> Bye
>>>> urs
>>>>
>>>> On 2015-12-22 16:05, [hidden email] wrote:
>>>>> The more I look at it that graphic really is odd.  After the first
>>>>> heartbeat call succeeds we clear the interval and no more requests
>>>>> should be made.  I noticed you have 8 heartbeat 200 heartbeat
>>>>> requests
>>>>> after the server comes up and 8 ReverseAjax.dwr calls.  Also, the
>>>>> calls are taking a super long time, that call should be taking a
>>>>> millisecond.  It is like you have 8 instances of engine.js running
>>>>> or
>>>>> something.  You mentioned you have 300+ classes exposed to DWR, if
>>>>> you
>>>>> reduce that to say 10, do you see the same behavior?
>>>>>
>>>>> On 2015-12-22 07:51, Urs Wolfer wrote:
>>>>>> On 2015-12-22 15:21, [hidden email] wrote:
>>>>>>> When in the attached graphic are you restarting the server?  You
>>>>>>> have
>>>>>>> a bunch of 200's and then _System.checkHeartbeat.dwr fails.  I
>>>>>>> would
>>>>>>> assume the server was taken down right before then.  If that is
>>>>>>> the
>>>>>>> case those pending ReverseAjax.dwr calls were there before the
>>>>>>> server
>>>>>>> went down unless you cleared your console.
>>>>>> The first red request (first visible request in the graphic) is
>>>>>> the
>>>>>> first request right after shutdown of the application. Then I have
>>>>>> restarted the application and a few requests failed until Jetty
>>>>>> was
>>>>>> accepting requests. All pending ReverseAjax.dwr were issued after
>>>>>> the
>>>>>> restart. After every pending  _System.checkHeartbeat.dwr which got
>>>>>> a
>>>>>> success-response, a ReverseAjax.dwr is started.
>>>>>> Bye
>>>>>> urs
>>>>>>
>>>>>>> On 2015-12-22 00:52, Urs Wolfer wrote:
>>>>>>>> Yes, it sounds also really strange to me. But I have builded DWR
>>>>>>>> by
>>>>>>>> myself in order to find the revision which introduced this
>>>>>>>> issue. And
>>>>>>>> after that, I have changed *only* the line noted below.
>>>>>>>> Some details about my setup:
>>>>>>>> * using Jetty 9.2
>>>>>>>> * ~300 DWR Java classes
>>>>>>>> * *not* using Spring or Hibernate
>>>>>>>> * DWR initialization time (including other services required by
>>>>>>>> DWR): ~20-20s
>>>>>>>> I have noticed that it is possible that two (or more)
>>>>>>>> ReverseAjax.dwr
>>>>>>>> are open after a restart for the same browser session (tab)
>>>>>>>> which
>>>>>>>> should not be possible IMHO. I think as long as a
>>>>>>>> __System.checkHeartbeat.dwr or ReverseAjax.dwr is pending (and
>>>>>>>> not
>>>>>>>> failed), no other request of the same type should be issued. You
>>>>>>>> can
>>>>>>>> see this behavior in the attached screenshot (screenshot
>>>>>>>> displays
>>>>>>>> situation right after restart; as you can see, there are many
>>>>>>>> ReverseAjax.dwr requests which all stay there (are re-newed)
>>>>>>>> after a
>>>>>>>> restart). Btw, I was not able to reproduce multiple
>>>>>>>> ReverseAjax.dwr in
>>>>>>>> an older snapshot (rc3-20120401).
>>>>>>>> Bye
>>>>>>>> urs
>>>>>>>> On 2015-12-21 16:21, [hidden email] wrote:
>>>>>>>>> The reporter of this issue must have messaged me privately and
>>>>>>>>> indicated the problem was with Jetty and their servlet config.
>>>>>>>>> I will
>>>>>>>>> look up his message and post it to the list.
>>>>>>>>> I also find it strange that this is the root cause of your
>>>>>>>>> problem.
>>>>>>>>> We have had thousands of downloads of 3.0 and no reports of
>>>>>>>>> something
>>>>>>>>> similar. The calls in question are heartbeat calls and should
>>>>>>>>> be
>>>>>>>>> executing extremely quickly, even if several of them queue
>>>>>>>>> there
>>>>>>>>> should not be a problem.  If you can minimize your code and
>>>>>>>>> send us a
>>>>>>>>> war that reproduces the problem that would be helpful.
>>>>>>>>> You are correct about the comments in engine.js.  I will update
>>>>>>>>> those
>>>>>>>>> and also clarify the retry docs a bit
>>>>>>>>>
>>>>>>>>> (http://directwebremoting.org/dwr/documentation/reverse-ajax/retry.html).
>>>>>>>>> On 2015-12-21 00:38, [hidden email] wrote:
>>>>>>>>>> I sufferred a problem which sounds almost like this problem.
>>>>>>>>>> After
>>>>>>>>>> updating to 3.0.1 (from 3.0.0-rc3-20120401), the application
>>>>>>>>>> did not
>>>>>>>>>> respond for a long time after a restart (sometimes several
>>>>>>>>>> minutes).
>>>>>>>>>> I tried to find out which commit introduced and was able to
>>>>>>>>>> identify
>>>>>>>>>> revision 3908 as breaking change for me. After looking into
>>>>>>>>>> all changes
>>>>>>>>>> done in this change, I've found out that the following change
>>>>>>>>>> introduced my problems:
>>>>>>>>>> -  dwr.engine._defaultRetryIntervals = [ 1, 1, 10 ];
>>>>>>>>>> +  dwr.engine._defaultRetryIntervals = [ 1, 3, 3 ];
>>>>>>>>>> There is a way to overwrite this setting in your application
>>>>>>>>>> code:
>>>>>>>>>> dwr.engine.setRetryIntervals([ 1, 1, 10 ]);
>>>>>>>>>> It sounds really strange that just this change causes such
>>>>>>>>>> issues.
>>>>>>>>>> While investigating a bit more, I have seen that the server
>>>>>>>>>> "bombs" the
>>>>>>>>>> server with a lot of "__System.checkHeartbeat.dwr" and
>>>>>>>>>> "ReverseAjax.dwr" requests in a short time while the server is
>>>>>>>>>> starting. All these requests keep the state pending and take
>>>>>>>>>> several
>>>>>>>>>> seconds to more than one minute to finish. While they are
>>>>>>>>>> pending, dwr
>>>>>>>>>> issues even more requests of this type.
>>>>>>>>>> Any ideas? I think you should not issue any more polling
>>>>>>>>>> requests while
>>>>>>>>>> requests are pending.
>>>>>>>>>> One minor note: the setting change should also result in an
>>>>>>>>>> update of
>>>>>>>>>> the comment: "Retry every 10 seconds when offline." (you have
>>>>>>>>>> changed
>>>>>>>>>> it to 3s).
>>>>>>>>>> Bye
>>>>>>>>>> urs
Reply | Threaded
Open this post in threaded view
|

Re: DWR 3.0 RElease and greater Init failure / Long delay on start.

uwolfer
On 2015-12-23 15:36, [hidden email] wrote:
> Ah, yes.  Thanks for tracking this down for us.  I have fixed this.
> Also, the suggestion you had on the Jetty cookie issue looked good.
> There is a new build out with both fixes -
>
> http://ci.directwebremoting.org/bamboo/browse/DWRTRUNK-ALL-657/artifact.
>  I am also pushing this build to Sonatype OSS repo if you use Maven.

Thank you very much. This release fixes all issues for me. Finally I
can update from an old 3.0 rc3 snapshot to the current version.

Bye
urs