Fully Unsupervised Learning for Continual Sequence when High Local Correlation

• Multi target estimation: baro
• Limited monte carlo with target space: baro
• Shared hidden space algorithm: baro

• Low dimension hidden space available: ljh93
  • cdim 100 hdim 400 - [reject]
  • cdim 20 hdim 400 - [accept], note: as decreasing think iteration, temperature softmax needs more selection
  • cdim 20 hdim 400 temperature 0.005 - [reject]
  • attention for announcing to model with more information
    • probability training with hidden space coefficient is superposition state
    • attention temperature inverse proportion to int(hdim/indim) that total iteration
      • other method that concern is attention with learnable parameter
      • if model can't infer well, we should raise temperature
• Check relay method effect with low dim hidden space: ljh93

• Hidden space regularization without relay mask: baro
• Hidden space regularization with relay mask replace with hidden space energy loss: baro
• Hidden space regularization without relay mask, hidden space energy loss and residual: baro
• Rollback
• Conclustion Loss error make weird loss graph and we success recover that but don't know why

• Loss with energy

• nan attention case check
• Select type 1 and type 2 attention and target