40,000 items in chinese language, Search Index was loading slowly and reply wrong results.

After Re-Index Advanced, the page was loading very slowly. (There are about 300k records in the table).
And after done, the result are all the same one.
Please check the attachment. The title and Search Index are all in Chinese, but I think you can see it's all the same.

The content of the table is good. Just the problem when click "Search Index".

The MYSQL slow query command is:

SELECT SQL_CALC_FOUND_ROWS ai.sid FROM kwtad_flexicontent_advsearch_index as ai JOIN kwtad_content as a ON ai.item_id=a.id ORDER BY a.title ASC LIMIT 0, 50

I don't know if it's a bug in newer version. I forgot when did I notice it. But I remember it started in previous version.

Or did I miss anything?

Thanks,
Lomu

Hello

please send me a PM with super admin access

Hello

After Re-Index Advanced, the page was loading very slowly. (There are about 300k records in the table).

The backend "search index" page will not effect frontend
- i have made some changes, now backend "search index" listing, should load 4 times faster

about updating the index you have more that 40,000 items , so it should take some time ...


- about correctly populating the index
chinese has no spaces , right ?

so you will need a word file, with all chinese words seperated by enter (new line)
otherwise the words are added as 1 big single word

can you provide such file ?

ggppdk wrote: Hello

After Re-Index Advanced, the page was loading very slowly. (There are about 300k records in the table).

The backend "search index" page will not effect frontend
- i have made some changes, now backend "search index" listing, should load 4 times faster

about updating the index you have more that 40,000 items , so it should take some time ...

OK I see. I know it takes time. I just feel it took too long than it should be.

- about correctly populating the index
chinese has no spaces , right ?

so you will need a word file, with all chinese words seperated by enter (new line)
otherwise the words are added as 1 big single word

can you provide such file ?

Basically there won't be space between Chinese words.
So what you ask for is a file with ALL Chinese words?
I can try to provide but just want to confirm.

Thanks,
Lomu

Hello

yes a word file like this

account
accountant
accounting


or a PHP class that does splitting of chinese text into words
i was thinking of this one:
www.phpbone.com/phpanalysis/

Hello

it seems that the word segmenter, that i mentioned above is best option

www.phpbone.com/phpanalysis/