Normalizing full-text search rank?

I read many good postings here about the weird logic of full-text
ranking, and I hope I finally got it (if it is possible at all ;).
Yet... We biult an application to search in our database in varoius
ways, incl. full-text search. I want to show users only "relevant"
matches, i.e. matches that score above some threshold. For other
(simple) search options one can easily return a score between 0 and
100% - for example, macthing 4 of 5 parameters searched will count
80%. How about full-text search, though? I can't simply normalize the
range 0-1000 saying rank 0 = 0%, rank 1000 = 100%, since even exact
matches (e.g. searching for "salt" in a row containing just that one
single word) return ranks as low as 160.
Any ideas, thoughts, considerations are welcome!
This can't really be done. The problem is that rank changes per search
arguement (SARG)/token. So searching on a word like "salt" might return a
set of relatively low rankings, but searching on a word which occurs
relatively rarely in your docuement set like "anthropormorphological" would
return a higher rank.
Words/SARGs/Tokens which occur relatively rarely have higher resolving power
than Words/SARGs/Tokens which occur frequently. For instance consider you
searching for a friend named John Smith. John Smith is a very common name,
so you need something that distinguishes your John Smith from all other John
Smith's - ie something relatively rare - like a middle name. Ranking weights
common Words/SARGs/Tokens lower, than Words/SARGs/Tokens which occur more
rarely as these Words/SARGs/Tokens has higher resolving power and hence
relevance.
Ranking is an attempt to measure relevance. It is a statistical measure and
is useful for ordering search results. Your actual "hit" could be the first,
second, of 15th result.
The best thing to do, is to order your results, pick the highest ranked
item, divide all other ranks by this value and multiply by 100.
Hilary Cotter
Looking for a book on SQL Server replication?
http://www.nwsu.com/0974973602.html
"istoynev" <istoynev@.hotmail.com> wrote in message
news:3b49ac82.0405142252.1a80edab@.posting.google.c om...
> I read many good postings here about the weird logic of full-text
> ranking, and I hope I finally got it (if it is possible at all ;).
> Yet... We biult an application to search in our database in varoius
> ways, incl. full-text search. I want to show users only "relevant"
> matches, i.e. matches that score above some threshold. For other
> (simple) search options one can easily return a score between 0 and
> 100% - for example, macthing 4 of 5 parameters searched will count
> 80%. How about full-text search, though? I can't simply normalize the
> range 0-1000 saying rank 0 = 0%, rank 1000 = 100%, since even exact
> matches (e.g. searching for "salt" in a row containing just that one
> single word) return ranks as low as 160.
> Any ideas, thoughts, considerations are welcome!
|||Thanks, Hilary!
I also thought about this trick, although it has bad bitter
performance impact, I am afraid... But it might be the only feasible
solution.
"Hilary Cotter" <hilaryk@.att.net> wrote in message news:<OiOcxB0OEHA.3300@.TK2MSFTNGP09.phx.gbl>...
> The best thing to do, is to order your results, pick the highest ranked
> item, divide all other ranks by this value and multiply by 100.
|||Again, performance with a TSQL solution would be expensive, if it was done on the client it would not be that expensive.
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
sql

No workaround for interactive htmlfragments at all?

Hi!
I've been really happy during the week-end as I finally found a way to do
long wanted things with RS 2000. Yesterday at 22 pm, I found out that
interactive HTML fragments isn't possible, after MANY hours of developing..
Yeah, go on.. call it bad planning. :(
Anyway, would it be possible to create a disassembly of the HTML4.0 renderer
using reflector and then modifying the code to fit my project? (Links and
toggles..)
Is it legal? Has anyone done this allready and published the code? Are there
any other possibilities other than URL access you guys would recommend?
Part of the reason I need fragments is that the security is based on a
hierarchy in a self-referencing sql table. I need to use a business/security
layer to create parameters for the reports which will be generated by a
web-service on a protected server. No way to limit the results with NT or
sql server security. (well, maybe sql server, but I have to re-engineer)
Hope for good help! :)
Lars-ErikCould you elaborate what exactly you want and what problem you are having?
"Lars-Erik Aabech" <larserik@.newsgroup.nospam> wrote in message
news:uMBw6fLnFHA.3256@.TK2MSFTNGP12.phx.gbl...
> Hi!
> I've been really happy during the week-end as I finally found a way to do
> long wanted things with RS 2000. Yesterday at 22 pm, I found out that
> interactive HTML fragments isn't possible, after MANY hours of
> developing.. Yeah, go on.. call it bad planning. :(
> Anyway, would it be possible to create a disassembly of the HTML4.0
> renderer using reflector and then modifying the code to fit my project?
> (Links and toggles..)
> Is it legal? Has anyone done this allready and published the code? Are
> there any other possibilities other than URL access you guys would
> recommend?
> Part of the reason I need fragments is that the security is based on a
> hierarchy in a self-referencing sql table. I need to use a
> business/security layer to create parameters for the reports which will be
> generated by a web-service on a protected server. No way to limit the
> results with NT or sql server security. (well, maybe sql server, but I
> have to re-engineer)
> Hope for good help! :)
> Lars-Erik
>|||Hi again!
I'm trying to create an asp.net form that uses some business logic to create
parameters for a report, then uses the web-service to get the html for the
report. The parameters should not be appended to a user visible/editable url
since that will compromise security. The WebService will be running on a
secure server, and the webservice proxy will impersonate an authorized
account. The report is built with recursive detail groups and a visibility
toggle.
This means I have to pick one or the other option. If I don't want to expose
the parameters, I have to use HTML fragments, else RS will append then to
all links, including targetting each link to the report server "url access"
page. I can create a report that has everything expanded and discard the
toggling functionality, but that'll mean pretty long reports.
Hope this helps explaining.
L-E
"Smith" <smithj@.online.nospam> wrote in message
news:ebqOKhMnFHA.320@.TK2MSFTNGP09.phx.gbl...
> Could you elaborate what exactly you want and what problem you are having?
> "Lars-Erik Aabech" <larserik@.newsgroup.nospam> wrote in message
> news:uMBw6fLnFHA.3256@.TK2MSFTNGP12.phx.gbl...
>> Hi!
>> I've been really happy during the week-end as I finally found a way to do
>> long wanted things with RS 2000. Yesterday at 22 pm, I found out that
>> interactive HTML fragments isn't possible, after MANY hours of
>> developing.. Yeah, go on.. call it bad planning. :(
>> Anyway, would it be possible to create a disassembly of the HTML4.0
>> renderer using reflector and then modifying the code to fit my project?
>> (Links and toggles..)
>> Is it legal? Has anyone done this allready and published the code? Are
>> there any other possibilities other than URL access you guys would
>> recommend?
>> Part of the reason I need fragments is that the security is based on a
>> hierarchy in a self-referencing sql table. I need to use a
>> business/security layer to create parameters for the reports which will
>> be generated by a web-service on a protected server. No way to limit the
>> results with NT or sql server security. (well, maybe sql server, but I
>> have to re-engineer)
>> Hope for good help! :)
>> Lars-Erik
>|||Hi,
It's true that you will loose the interactive capabilities using HTML
fragments. An alternative would be using the ReportViewer control which
will not expose the parameters in address bar, but the parameters would
still be seen when you right-click the page and select 'View Source'.
Sincerely,
William Wang
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
This posting is provided "AS IS" with no warranties, and confers no rights.|||Hi!
Thank you for answering. Do you have any views on my actual question?
> Anyway, would it be possible to create a disassembly of the HTML4.0
> renderer
> using reflector and then modifying the code to fit my project? (Links and
> toggles..)
> Is it legal? Has anyone done this allready and published the code? Are
> there
> any other possibilities other than URL access you guys would recommend?
Lars-Erik
"William Wang[MSFT]" <v-rxwang@.online.microsoft.com> wrote in message
news:qdHZ%23OmnFHA.940@.TK2MSFTNGXA01.phx.gbl...
> Hi,
> It's true that you will loose the interactive capabilities using HTML
> fragments. An alternative would be using the ReportViewer control which
> will not expose the parameters in address bar, but the parameters would
> still be seen when you right-click the page and select 'View Source'.
> Sincerely,
> William Wang
> Microsoft Online Partner Support
> When responding to posts, please "Reply to Group" via your newsreader so
> that others may learn and benefit from your issue.
> This posting is provided "AS IS" with no warranties, and confers no
> rights.
>|||To the best of my knowledge, reverse-engineering MS code is illegal.
Actually it's not recommended to pass sensitive info as part of a report
parameter to begin with. Tudor's blog entry has addressed this issue:
http://blogs.msdn.com/tudortr/archive/2004/07/20/189398.aspx
Sincerely,
William Wang
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
This posting is provided "AS IS" with no warranties, and confers no rights.|||Thank you! This was pretty helpful. :)
I'll consider running Reporting Services on the front-end server using a
custom authentication module.
Lars-Erik
"William Wang[MSFT]" <v-rxwang@.online.microsoft.com> wrote in message
news:rhnK2AloFHA.588@.TK2MSFTNGXA01.phx.gbl...
> To the best of my knowledge, reverse-engineering MS code is illegal.
> Actually it's not recommended to pass sensitive info as part of a report
> parameter to begin with. Tudor's blog entry has addressed this issue:
> http://blogs.msdn.com/tudortr/archive/2004/07/20/189398.aspx
> Sincerely,
> William Wang
> Microsoft Online Partner Support
> When responding to posts, please "Reply to Group" via your newsreader so
> that others may learn and benefit from your issue.
> This posting is provided "AS IS" with no warranties, and confers no
> rights.
>|||Hi Lars,
I'm glad to hear the information provided is helpful. Please let me know if
you have any other concerns, or need anything else.
Sincerely,
William Wang
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
This posting is provided "AS IS" with no warranties, and confers no rights.