Skip to content

Order of operations for applying filters. #161

Closed

Description

@rbdavid

Major question/concern: Order of operations for applying filters will affect the results. Currently, the filters are applied in steps Fraction, Fragment, and then Taxonomy. Which sequences that pass the Fraction filter is a bit random and certainly uncaring to the biology; that filter just blindly removes based on a modulo boolean check. Whatever sequences remain may still not pass the other filters. This has the potential to really hamstring an analysis.

Applying the Fraction filter last, would enable the other filters to thoroughly remove the biologically uninteresting sequences, leaving behind a set of sequences that we know the user is interested in, from which the Fraction filter can remove sequences in a more random, uncaring fashion.

But, changing the order of filtering operations will/may return different results compared to those from the EFI v1 code base.

Originally posted by @rbdavid in #151 (comment)

I would put this as a low priority question to ask John/Remi later this summer. It can be added as a low priority issue if you want.

Originally posted by @nilsoberg in #151 (comment)

Activity

  1. added theissue type on Apr 17, 2025
  2. rbdavid commented on Apr 17, 2025

    @rbdavid
    ContributorAuthor

    Refering to the Fraction filter specifically:

    Given the same ordering of IDs, this [filter] will be deterministic. But, if the ordering changes (whether due to change in the EFI-DB version or some nuance of querying), recreating the same fractioning of the input IDs may not be recovered exactly. Should we add some text disclaimer about this on the frontend?

    Originally posted by @rbdavid in #151 (comment)

  3. rbdavid commented on Apr 17, 2025

    @rbdavid
    ContributorAuthor

    Referring to the Fraction filtering:

    What happens if the $opts->{fraction} value is larger than the total number of members in the family? Do we throw a warning message saying only one sequence was gathered from the set?

    Originally posted by @rbdavid in #151 (comment)

  4. removed theissue type on Apr 24, 2025
  5. locked and limited conversation to collaborators on Jul 29, 2025
  6. converted this issue into a discussion #195 on Jul 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions