Group: CiteULike-discussion - Forum Thread

Topic: Bug reports

Importing my large BibTeX file fails

I would like to import my BibTeX file which contains roughly 13000 entries. There appears to be a 5,000 limit imposed per import, so I made the file smaller than that. The file is transferred to citeulike but then my browser just hangs forever.

Posted by alterego on 2008-05-05 15:56:08.

This thread is closed

7 replies. Login or join this group to post to this thread.

The BibTeX import is quite slow, so you tend to hit problems like web server timeouts - there's no coded-in limit to how large the files can be. If you can bear to, split the file into about 10 equal sized chunks and import them.

Posted by cjhall on 2008-05-05 17:10:13.

There is a coded limit on how large the files can be. It gave me an explicit message that it had hit the BibTeX limit of 5000 and that it was a fatal error.

Posted by alterego on 2008-05-05 18:16:24.

Ahhhh - that's BibTeX itself then - we use it pre-process the files so that things like macros get dealt with properly - not our code. I know it makes no difference to you - it still just a work... Try it in smaller pieces.

Posted by cjhall on 2008-05-05 18:50:18.

This would be one solution to my current problem too, with only 700 entries. How would one go about splitting the Bibtex file, please? There isn't an "export in parts" function.

Posted by GI-Sci on 2008-05-21 13:36:23.

Hmmm. 700 entries should import easily. Could you email it to support at citeulike dot org. At the very least, I'll send it back to you in small pieces...

Posted by cjhall on 2008-05-21 14:21:16.

It said it was over 5000 lines... I didn't catch the response here before responding on the other thread, both being towards the same end. But surely I'll email the file directly. And thank you again.

Posted by GI-Sci on 2008-05-21 22:04:31.

I've tracked down some of the size issues with BibTeX imports. I'm afraid they are all caused by BibTeX itself - it seems to have some compiled-in limitations which are:

Maximum field size (or maybe it's entry size) of 5000 characters
Some internal hash table which seems to limit your file to about 4000 entries

These are examples of the error messages:

Your field is more than 5000 characters---line 129 of file test.bib
Sorry---you've exceeded BibTeX's hash size 35307

In the first case, you just have to trim fields to make them shorter - not nice. In the second case, just keep the files to less than about 4000 entries, and all should be well - if not, it's a problem in our code and, therefore, fixable. For what it's worth the RIS import has no such limitations.

Posted by cjhall on 2008-05-21 22:57:27.