Using extra computer resources

faq
flatforty
contribute
subscribe
configure
search
rdf

main
parent

Using extra computer resources
by Waldo Bastian on Thursday 03/Mar/2005, @03:13

It should be noted that CPU processing power has increased dramatically in the last 9 years but that harddisk access times haven't. KDE performance is limited mostly by the latter and adding background search indexing is going to hurt there because it will make the harddisk access times worse.

Even despite the increase in CPU processing, drawing in e.g. Konsole manages to keep even the fastest CPU's busy because of Qt's poor support for fixed width fonts.

So if you like challenges: redesign Konsole to work with variable width fonts and still have your "ls -al" output line up.

Extra cookies are available for bidi-support.

Related Links

·   Articles on Community and Events
·   Also by Waldo Bastian
·   Contact author

The Fine Print: The following comments are owned by whomever posted them.
( Reply )

Re: Using extra computer resources
by Simon Edwards on Thursday 03/Mar/2005, @03:54

It should also be noted that:

a) if you index the contents of file just after it has been modified, then it is probably still in the system's disk cache.

b) Most desktop hard disks are not kept busy 100% of the time. They are often doing nothing. This wasted time can be used for indexing. The inder will also need stop what it is doing when something more important (like the user) wants to use the hard disk.

There is no reason for search indexing to hurt.

Before everyone gets the wrong impression about Scott's work, content indexing *is not* what it is all about. Although it looks like it will play a part, the idea is to record the context and 'links' between things and documents that users create implicitly by just using their desktop and applications.

just IMHO of course.

--
Simon

[ Reply To This | View ]

Re: Using extra computer resources
by koos on Thursday 03/Mar/2005, @08:59

> There is no reason for search indexing to hurt.

Is there already a way (or always has been w/o me knowing it :-)) to do disk indexing w/o trashing the disk caches?

[ Reply To This | View ]

Re: Using extra computer resources
by Simon Edwards on Thursday 03/Mar/2005, @10:18

> Is there already a way to do disk indexing w/o trashing the disk caches?

nope,

but you don't have to constantly thrash the hard disk either. That is only needed when building the _initial_ index.

--
Simon

[ Reply To This | View ]

Re: Using extra computer resources
by Aaron J. Seigo on Thursday 03/Mar/2005, @15:29

and even the initial indexing can probably be mitigated somewhat by being run as a lazy background process.

[ Reply To This | View ]

Re: Using extra computer resources
by koos on Thursday 03/Mar/2005, @16:31

I don't understand what you mean with thrash the hard disk. You said that it should run if hdd is idle and it wouldn't hurt performance. But in the morning, after updatedb has ran, my system is a lot slower (that's what I meant with the disk cache being trashed). Well at least on linux, their is no API that I know of to skip disk caching while indexing (or eg. playing a DVD).
So, how do you manage to not hurt performance then?

[ Reply To This | View ]

Re: Using extra computer resources
by Scott Wheeler on Thursday 03/Mar/2005, @18:16

It's becoming a requirement of modern kernels to have useful (i.e. not dnotify) mechanisms for notifying user-space of file system changes. At least on Linux there's a reasonable chance that by the time this stuff is ready for the mainstream that inotify will also be something we can rely on.

We'll see. But these really aren't the issues that we're worrying about now, premature optimization being the root of all evil and all that.

[ Reply To This | View ]

Re: Using extra computer resources
by Ngigi Waithaka on Thursday 03/Mar/2005, @07:09

Why then can we just hack on the QT implementation of the fonts? This would be more efficient than a picemeal approach!

[ Reply To This | View ]

Re: Using extra computer resources
by Aaron J. Seigo on Thursday 03/Mar/2005, @15:36

> adding background search indexing is going to hurt there because it will make
> the harddisk access times worse

it seems that we will likely only have to index files "relevant" to desktop search. so... we likely won't be indexing things like the http cache, or system directories, or tmp directories, or ... the number of "relevant" file writes and locations is, hopefully, manageable.

by plugging support for such things into the file dialogs, we should be able to catch many such file writes quickly.

this doesn't mean it's without yet-to-be-solved challenges such as which locations to index by default, or how to pick up changes that were made on the file system but not through the system (changed file notifications, basically). there's also the factor of how up to date the search needs to be in relationship to the file system.

when someone saves 100 files in one go to the filesystem, do they need to be indexed immediately? what are the "rules" that govern this?

when someone removes 100 files from the filesystem, how quickly do they need to be removed from the index?

etc... the file indexer will be an interesting project in its own right. =)

[ Reply To This | View ]

The Fine Print: The previous comments are owned by whomever posted them.
( Reply )


		"Sorry, security is not optional." -- Waldo Bastian
KDE®, "K Desktop Environment", "KDE Dot News", "got the dot?" and the KDE Logo® are trademarks or registered trademarks of KDE e.V. in the European Union, the United States and other countries. All other trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the poster. The rest: Copyright © 2000-2008 KDE e.V. for The KDE Project. For further information or comments on this site, please contact the Webmaster.