There are many tutorial using add_tokens(special_tokens=True), but I read the source code, and find that add_special_tokens will do more thing than add_tokens, Which is prefer?
There are many tutorial using add_tokens(special_tokens=True), but I read the source code, and find that add_special_tokens will do more thing than add_tokens, Which is prefer?