Hello Joe, Lee or anyone else in this community,

First of all thank you for giving me the heads up on the fact that no matter what the host company might have suggested to talk with their technitions again that was a very smart plan.  I did this and I learned this time that yes the Web application was being shut down by them.

It appears I have reached their 100 MB quota as far as the web application goes.  It appears if the web app uses more then 100 MB of memory they shut down or recycle a web app.

What would cause dotnetnuke to consume over 100 MB (really that doesn't seem to be all that much in this day an age anyway) could it have been the combination of the upgrade to dnn 4.8 from dnn 4.5.1 and the migration from sql 2000 to 2005 that is causing the rise in memory usage by the web app?

Also is their anything I could do on my end in terms of web app optimization so that I can try to reduce the amount of memory being used by dotnetnuke.  At this time I don't have any video stuff running that would drain memory or anything else that I can think of that in itself would be a memory hog.

I am going to talk with my host company and see if their is a way to buy more ram usage for my dotnetnuke site so we don't get these web app shut down errors every 2 minutes or so but I would stilll anxious to know what if at all anything could be done from my side to reduce the amount of memory the dotnetnuke is using from the servers memory bank thanks

Bo

You might want to try disk caching rather than memory caching ... assuming that's what you are doing ...

I just looked at the host performance area and it was already on disk caching. I know I posted this question on the last to posts regarding schedule failures but do you think that having the dotnetnike scheduled services is causing the chaos as far as memory is concerned.

I tried on my localhost turning these services on and seeing them fail just like the production server and the sqlservr.exe process was initially doing about 16mb and after turning on the schedules it was doing almost 100 MB as it relates to memory usage but it isn't totally fair assessment as I turned on the schedules and left my computer for a couple hours so I wasn't able to test it with the schedules turned off I probably need to do that to but something is hogging lots of memory and I don't know what if it is do to the upgrade and migration or something else but seems like many others would complain if sql2005 and dnn 4.8 ended up hogging a lot more memory thanks

Bo

You might want to turn on scheduled items one at at time to see which one if the "hog."

But, I'm not sure that I would agree that 100 MB is a lot of memory. Is there a chance that you could get your hosting provider to give you a larger quota?

I tried that Joe and apparently it is some system setting that cannot be ammended according to the host provider discountasp.com.

Here is their very words from an email I received concerning this issue:

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

"I can't really offer any suggestions on reducing your web application's overhead since it is outside the scope of technical support
 





so the only suggestion I could make is to fully evaluate your application to see which areas cause excessive memory consumption.

Unfortunately, the 100 MB memory threshold can't be adjusted as it is a system wide setting across all web servers and there aren't
 


any addons that allow for additional application memory usage.

The event log only records that the application pool was recycled and just a generic reason why, there isn't any specific data
 



that would be helpful."


----------------------------------------------------------------------------------------------------------------------------------

I'm going to try and turn on the search engine service and see if that is what is doing it.  It appears that when most services

are turned off I only get 5 pages of errors in 12 hours not 20 pages but I'll do as you said starting first and foremost with
the searchengine service that indexes my search engine on dotnetnuke.  This schedule by the way fails like all the rest but
that is another issue altogether thanks for your help in these questions

Bo

 

 

I tried that Joe and apparently it is some system setting that cannot be ammended according to the host provider discountasp.com.
Here is their very words from an email I received concerning this issue:

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
"I can't really offer any suggestions on reducing your web application's overhead since it is outside the scope of technical support

so the only suggestion I could make is to fully evaluate your application to see which areas cause excessive memory consumption.

Unfortunately, the 100 MB memory threshold can't be adjusted as it is a system wide setting across all web servers and there aren't

any addons that allow for additional application memory usage.

The event log only records that the application pool was recycled and just a generic reason why, there isn't any specific data

that would be helpful."


----------------------------------------------------------------------------------------------------------------------------------

I'm going to try and turn on the search engine service and see if that is what is doing it. It appears that when most services

are turned off I only get 5 pages of errors in 12 hours not 20 pages but I'll do as you said starting first and foremost with
the searchengine service that indexes my search engine on dotnetnuke. This schedule by the way fails like all the rest but
that is another issue altogether thanks for your help in these questions

Bo

It appears Joe and Lee,

That the memory "hog" or at least the thing that is shutting down the web app so often is this service:

------------------------------------------------------------------
DotNetNuke.Services.Search.SearchEngineScheduler, DOTNETNUKE
------------------------------------------------------------------

This is the service that re-indexes the search engine for the site so that when an annonymous visitor or registered types in a word within the Dnn search box it allows the visitor to find the latest things that had been added to the site within such and such a time frame isn't that right/

If this is the case is their anything to your knowledge that could be done so thatthis job could execute properly and not fail all the time or is it failing because the host company has put that 100 MB cap on the site and hence these services that go out and purge log buffers and site history's and update search engines can't for some reason or another operate though it seems when the host decides to recycle the web app from time to time people just get a server error but I wonder if that 100 MB cap is part of the problem that isn't allowing these secheduled services to execute properly.

Have you guys ever had issues with your scheduled services via host>schedule area.

Also I know I asked Joe and Lee about the host provider they are using but I want anyone who might be viewing this discussion to let me in on what host provider they are currently on and if their is any kind of fixed memory cap that is either causing issues in your dotnetnuke installation or just simply if your hosting company has created a cap that cannot be raised.

I am thinking that if discountasp.com is the only service that puts this kind of cap on things that it maybe useful for me to change host companies and find one that doesn't put these kinds of restrictions on their customers.

Thanks for your help,

Bo

Hello,

Oh yes the search provider - I have experienced that problem on DNN Creative. - unfortunately the core search engine is not very efficient and it needs reworking. I had to stop my search provider way back in something like issue 8 / 9 so most of the site will not appear in my search results! - the problem is that when your site reaches a certain size the search engine scheduler cannot index the entire site and can cause memory / cpu maxing problems.

I am currently looking into implementing the Venexus search engine module as this is a much more efficient way of implementing search within your site.

I am on shared hosting and I do not know what the memory cap is on my sites. - you can have a read about various hosting providers here: DotNetNuke® Hosting Providers

Hello Lee,

Just opening up this discussion once again since my client has urged me without a shadow of a doubt to find a feasible solution to this search privider services issue.

I have noticed that when I shut off this provider these web sites going down were slim if at all so I know your response to this issue was accurate and I just wanted to ask you if that search module you were looking into has been any good for you.

If you have any other modules or ideas of getting large sites fully indexed without the use of the search engine service I would love to hear about it since this has just become high priority in my to do list for our website thanks again.

 

Bo

This is just a side issue but just wondering Lee or John is their a way to see all the questions I have posted on my account. The reason I ask is because I first opened this discussion a couple months or so ago and haven't really had a need to persue it further until just recently and luckily by clicking on page 10 I happened to find this question and so could persue this question instead of writing a new one but there may be issues that reserface a lot longer from now and I certainly will not find that issue (unless someone else found it and responsed to it) amongst all the issues that may have arisen within a year or two time frame so as such I was wondering if in my account I could isolate all the questions I brought up so as to be able to easily find a stalled issue that may need a relook.

Thanks,

Bo

Just to add one more thing to this discussion is there anyway of finding or buying a skin object that can be a substitute for this search engine service that seems to not handle resources well for larger sites?

I know we discussed the possibility of a perticular module but as you read these posts if you know of any skin objects can also serve this purpose of internal searching in indexing that would be great.

Also what would you see as the better method of going about this internal search substitute as a skin object or as a module.

Thanks again for reading and in advancef or replying.

Bo

If you click on your name next to a post, you should be taken to "your profile." There is a link there that you can click to view your posts.

As for the other questions, I'm not sure that's the real issue. Make sure that your caching of pages is to disk and not to memory. That might help with the memory usage issue. I don't quite see how search indexing eats up that much memory. It could, but I'm skeptical.

I have only briefly experimented with the Venexus search, but it is much more efficient than DNN core search as it uses full-text indexing, plus if I recall correctly, I think they do provide a search skin object for it, so it can be integrated right into a skin just at the core search.

The search module is also on my to do list for experimenting with and implementing, so if you go down this route let us know how you get on. Thanks,

Hello Lee,

I have done my own research concerning this search module from venexus by calling them. They have their own toll free number if you click on the contact us link and you can get a hold of one of their technitions starting at 9:00 AM EST or 4:00 AM England time. They are opened 12 hours a day Monday-Thursday I believe it is it might be thru Friday but certianly thru Thursday.

Anyway what I have learned from my correspondance was the following:

It is recommended that you have at least 512 MB at your disposal to utilize their search engine module. They told me it may work with 100 MB but it would be very sluggish and slow. In my case it probably would attempt to use more than 100 MB of RAM resources hence taking down my site. At any rate as far as the search features go they are extensive and according to them if you have at least 512 MB of RAM on your sql server2005 to work with you can search very fast and efficient.

What this search module will do is quickly and efficiently search content from your core modules on DNN (i.e. text/html, event, etc...) What it won't do is be able to find content from those third party modules you have installed on your Dotnetnuke Installation unless their is an RSS feed added to it since this search module finds rss feeds. It also will find excel and word documents. I believe it will also find powerpoint documents as well but I would double check on that if finding powerpoint documents is an issue for your dotnetnuke site.

Apparently you can enable the capability for global crawling which simply means it can crawl the entire internet if you want it to howbeit it may find much unwanted trash if you do that.

You also have a filtering capability which boils down to an approval process of items the search aggregaitor located and decide weather you want particular items to be able to be found by your visitors or not. I guess it is kind of like an approval system like one might have for posts but this would be approving indexed pages to be visable to a visitor via typing in a keyword in the search box.

It was an itneresting concept to me but that was pretty much the extent of it. If you are thinking of going for This module I would highly recommend exploring the possibility of a 12 month subscription for around $175. It may be a tiny bit more but not much. This would give you access to all their modules and skin objects for 12 months. Included in this would be updates for the time frame you are subsribed without extra costs.

The searchengine module at least their pro version was about $200 and acccording to my conversation with one of their representatives it appears that this 12 month subscription to all their modules and skins would give you more bang for the buch as it would give you the opportunity to download all the modules and skins they have for around $20 less than it would cost to by just that one search engine pro module.

They told me they were currently working on their pricing structure so if you are serious about using this search module as an alternative I would highly recommend you get on the stick right away before theiy have changed their pricing structure.

Really it could mean prices are being nogotiated to be lower but I have a feeling that this is not the case most likely they are working on a pricing structure that will include an increase in cost but while they are in the process of hashing this out it would be good to research it out for yourself and perhaps invest int he module or subscription now before prices possibly go up.

For me I will not get this module at this time do to the bottleneck I face with my hosting company only allocating 100 MB for my use but for those who are not under this bottleneck this might be a very wise investment especially if you are having issues with your DNN core search engine services causing issues of slowness or site recycling issues.

Anyway just figured I would give you my evaluation now so you can look at it and perhaps take a look at this site.

You can find information about their modules and about them at the following address: http://www.venexus.com/

Good luck to all who are trying to investigate alternatives to your current internal search situations.

Bo

Hello,

That's great, thanks for taking the time to post all of this information, I'm sure people will find this useful,

Thanks,

Hello I need to reply to this post because I have discovered something about DNN search solution and need some insight. We can't go with the option above do to the price and do to the fact that after paying it we are not guaranteed that it will deliver do to our RAM cap. However I have just visited the dotnetnuke forum and saw a post concerning the DNN search issue.

It said that you should be running in timer mode which is found in the host settings do to the fact by doing this the search service will run on another thread thus improving performance though obviously DNN Search is still not the best solution for large websites.

I have done this and have my search service running once every 24 hours and had a retry of once every 12 hours which I recently disabled. However I wanted to post the history I have received from the search service in hopes that maybe you guys have a fix for the problem. The search goes for I think it was 72 seconds at max and every time weather 72 seconds, 40 seconds, or 30 seconds I received an abort message in the history of this service take a look.

Description Duration (seconds) Succeeded Start/End/Next Start
DotNetNuke.Services.Search.SearchEngineScheduler, DOTNETNUKE
EXCEPTION: Thread was being aborted. 401.86 False S: 5/22/2008 11:05:56 PM
E: 5/22/2008 11:12:38 PM
N: 5/23/2008 11:05:56 AM
DotNetNuke.Services.Search.SearchEngineScheduler, DOTNETNUKE
72.374 False S: 5/22/2008 11:07:08 AM
E: 5/22/2008 11:08:20 AM
N: 5/23/2008 7:53:27 AM
DotNetNuke.Services.Search.SearchEngineScheduler, DOTNETNUKE
EXCEPTION: Thread was being aborted. 31.047 False S: 5/22/2008 11:05:45 AM
E: 5/22/2008 11:06:16 AM
N: 5/22/2008 11:05:45 PM
DotNetNuke.Services.Search.SearchEngineScheduler, DOTNETNUKE
31.67 False S: 5/22/2008 11:04:44 AM
E: 5/22/2008 11:05:16 AM
N: 5/23/2008 7:53:27 AM

Besides disabling the retry intervals is their anything else you guys would suggest that could at least cause this search engine to index some of the site before aborting. What causes it to abort in the first place. I know this solution is certainly not efficient but if it can work on timer mode it is better than no solution.

Thanks,

Bo

P.S. if you are curious as I was how to change your setting to timer mode here is how it is done.

1. Go to host settings>other Settings

2. Next look for the schedule mode property and in the field to the right where by default it says Request Method click on the arrow by the field and choose Timer Mode and than of course click update. What this does according to the DNN help is the following:

The Timer Method maintains a separate thread to execute scheduled tasks while the worker process is alive. Alternatively, the Request Method executes tasks when HTTP Requests are made. You can also disable the scheduler by selecting Disabled.