#6 sentences.corpus is rewritten every time Kaylee runs

Closed
opened 7 years ago by clara · 1 comments
clara commented 7 years ago

It seems silly to regenerate this file every time Kaylee starts. If the voice commands haven’t changed, the file will be exactly the same as before, but we created extra I/O. Sure the file won’t be huge, so it’s not a big deal, but it’s a silly inefficiency when we’re already hashing files to see if things have changed.

A possible solution would be to generate the file in memory, and if the hash differs from the stored one, write the generated file out to disk. This would allow us to do things like remove duplicate words, though the lmtool doesn’t seem to care if we have duplicate words so why should I?

It seems silly to regenerate this file every time Kaylee starts. If the voice commands haven't changed, the file will be exactly the same as before, but we created extra I/O. Sure the file won't be huge, so it's not a big deal, but it's a silly inefficiency when we're already hashing files to see if things have changed. A possible solution would be to generate the file in memory, and if the hash differs from the stored one, write the generated file out to disk. This would allow us to do things like remove duplicate words, though the lmtool doesn't seem to care if we have duplicate words so why should I?
clara commented 7 years ago
Owner

Note: According to this CMUSphinx Wiki article, we really shouldn’t remove duplicate words from the corpus, and in fact we should do something smarter about numbers than we’re doing now. That should be a separate issue, though.

Note: According to [this CMUSphinx Wiki article](http://cmusphinx.sourceforge.net/wiki/tutoriallm), we really shouldn't remove duplicate words from the corpus, and in fact we should do something smarter about numbers than we're doing now. That should be a separate issue, though.
Sign in to join this conversation.
No Milestone
No Assignees
1 Participants
Due Date

Dec 31, 0000 Overdue

Dependencies

This issue currently doesn't have any dependencies.

Loading…
Cancel
Save
There is no content yet.