Which objects can I use in a finalizer method?

I have a class that should delete some file when disposed or finalized. Inside finalizers I can't use other objects because they could have been garbage-collected already.

Am I missing some point regarding finalizers and strings could be used?

UPD: Something like that:

public class TempFileStream : FileStream
{
    private string _filename;

    public TempFileStream(string filename)
        :base(filename, FileMode.Open, FileAccess.Read, FileShare.Read)
    {
        _filename = filename;
    }

    protected override void Dispose(bool disposing)
    {
        base.Dispose(disposing);
        if (_filename == null) return;

        try
        {
            File.Delete(_filename); // <-- oops! _filename could be gc-ed already
            _filename = null;
        }
        catch (Exception e)
        {
            ...
        }
    }
}