Wind rose - Sydney Observatory Hill

[FedericoTartarini]

Describe the bug
The data for this location are incorrect. I am not sure if other locations are also affected and how we can fix this and similar problems

Screenshots

[ziqi0067]

@FedericoTartarini ：
I have now completed the self-inspection phase and hotfix for this issue. However, I still haven't resolved the hourly wind speed test. For the annual wind speed test, I discovered issues with the original data using Debug.
I suspect there might be an issue with the EPW source file, but I'm unsure how to open it. I've attached a screenshot of my current process for reference and would appreciate any suggestions.

Wind_Problems.docx

[FedericoTartarini]

@ziqi0067 I could not fully understand your message. However, yes, I confirm that the issue is with the EPW source file. If the file is corrupted or it does not contain valid data, we should either display a warning to the user or not load the charts.

[ziqi0067]

Okay, so you mean this page displays a warning or doesn't need to be loaded, right? @FedericoTartarini

[FedericoTartarini]

Sorry but I do not understand your messages, could you please provide more context and clearly explain what do you mean?

[ziqi0067]

What I mean is, I agree with your point. I'd like to implement a pop-up or prompt to inform users that the data here is incorrect and they cannot click through to that location, or you could understand it as being unable to load the page. @FedericoTartarini

[ziqi0067]

Add a validation function to check locations where Wind data is problematic.

def _wind_looks_broken(df) -> bool:

    if df is None:
        return True
    d = pd.DataFrame(df)
    if d.empty:
        return True

    spd_col = ColNames.WIND_SPEED
    if spd_col not in d.columns:
        return True

    spd = pd.to_numeric(d[spd_col], errors="coerce").dropna()
    if spd.empty:
        return True

    frac_le_33 = (spd <= 3.3).mean()
    frac_ge_55 = (spd >= 5.5).mean()

    if frac_le_33 >= 0.95 and frac_ge_55 <= 0.001:
        return True

    if spd.max() <= 3.0:
        return True

    return False

New: popup box when wind data is not available

 dmc.Modal(
                id="wind-alert-modal",
                opened=False,
                title="⚠️ Wind data unavailable",
                centered=True,
                zIndex=1000,
                size="lg",
            ),

Set a callback function for warning messages

@callback(
    [
        Output(ElementIds.WIND_ALERT_MODAL, "opened"),
        Output(ElementIds.WIND_ALERT_MODAL, "children"),
        Output(ElementIds.WIND_PAGE_BODY, "children"),
    ],
    # Directly monitor data so that upon initial page load or site switching, immediate determination can be made.
    [
        Input(ElementIds.ID_WIND_DF_STORE, "data"),
        Input(ElementIds.ID_WIND_META_STORE, "data"),
    ],
    prevent_initial_call=False,
)

Dash Callback Functions

def guard_wind_page_by_content(df, meta):
    broken = _wind_looks_broken(df)

    site = None
    if isinstance(meta, dict):
        site = meta.get("location_name") or meta.get("site") or meta.get("station_name")
    site_label = site or "This site"

    msg = messages_alert.get(
        "not_available", "The EPW for this location is not available"
    )

    if broken:
        body = dmc.Stack(
            [
                dmc.Text(f"{site_label}: {msg}", size="md"),
                dmc.Text(
                    "Wind data look abnormal for this location. The page is disabled.",
                    size="sm",
                    c="dimmed",
                ),
            ],
            gap="xs",
        )

        return True, body, []

    return (
        False,
        no_update,
        [
            title_with_link(
                text="Annual Wind Rose",
                id_button=IdButtons.WIND_ROSE_LABEL,
                doc_link=DocLinks.WIND_ROSE,
            ),
            dcc.Loading(
                type="circle",
                children=dmc.Stack(id=ElementIds.WIND_ROSE, w="100%"),
            ),
            dcc.Loading(
                type="circle",
                children=dmc.Stack(id=ElementIds.WIND_SPEED, w="100%"),
            ),
            dcc.Loading(
                type="circle",
                children=dmc.Stack(id=ElementIds.WIND_DIRECTION, w="100%"),
            ),
            seasonal_wind_rose(),
            daily_wind_rose(),
            custom_wind_rose(),
        ],
    )

The above is the code block I added. Next, I will demonstrate the warning dialog box on the interface. One point I want to clarify is that the code block I added does not affect other pages, and users can still access other pages like Temperature and Outdoor normally. The page styles in the screenshot appear unusual because I haven't merged the latest repository version yet, as our team is currently focused on modifying the NavBar section.

[ziqi0067]

@FedericoTartarini Please take a look at my revised proposal for this issue.

[giobetti]

Thanks for looking into this. What is the behavior on other pages that rely on wind speed data? This is especially important for the outdoor comfort tab. Best Regards, Giovanni

[FedericoTartarini]

@ziqi0067 thank you for looking into this. I like that you started proposing some sulutions but as @giobetti mentioned you will need to look at the problem more in detail:

1. What is the behavior on other pages that rely on wind speed data?
2. The tests you are using to check if the wind speed column is corrupted are not comprehensive enough and may cause some false positives.
3. what about if the other columns are corrupted, we need to define a strategy that check that for example also the t, rh, ... columns have valid data.

@giobetti sorry to bother you, however, if possible do not reply to the email you receive directly otherwise it will add all the previous conversation to the issue making the comment very long and hard to read. Alternatively make sure that if you reply to the issue please manually delete the previous conversations.

[ziqi0067]

1. Regarding the first issue, the current wind speed data anomaly is only displayed on the wind speed page. I'm not quite sure what you mean. Are you suggesting that if there are problems with the wind speed data on other pages, users should also be warned that they cannot proceed? If that is the case, undertaking this task personally would involve a significant amount of work. Do you have any suggestions for optimising this?
2. Regarding your comment about the wind speed column checks being incomplete, could you specify which aspects are lacking? Might you propose some solutions?
3. Concerning the third point, are you proposing to define a global column-checking function?
   @FedericoTartarini

[FedericoTartarini]

1. we should check that the data are okay when we load them after a user selects a location and show warning messages if there is an issue with the data. Similar to what you have done but moving the login to when we load the data in se select page.
2. this the part I would like you to work on. This will be your original contribution. We need to check for example that no data is missing, not all data are the same, ....
3. yes, the same logic in point 2 should be applied to all the columns. Once all the columns are checked we can then show warnings to the users if needed.